KeyHandler

The KeyHandler is the main interface with other key files that point at measurement sets, imaging recipes etc. The pipeline is run by instantiating the KeyHandler, and then passing it to the other handlers.

class phangsPipeline.KeyHandler(master_key: str = 'key_templates/master_key.txt', dochecks: bool = True)

Class to handle data files that indicate the names and data sets associated with reducing a (generally) ALMA imaging project.

The KeyHandler generally just gets passed the path to the master key file (which tells the pipeline where to look for other relevant files, imaging recipes etc.), and then will be passed to other handlers for data processing.

Parameters:

• master_key (str, optional) – Path to master key file. Defaults to ‘key_templates/master_key.txt’.

• dochecks (bool, optional) – Whether to check for missing files. Defaults to True.

build_key_handler(master_key=None)

Construct the key handler object.

check_dir_existence(imaging=True, postprocess=True, derived=True, release=True)

Check the existence of the directories for imaging and post-processing.

check_key_existence()

Check file existence for the input keys defined in the master file.

check_ms_existence()

Check that the measurement sets in the ms key all exist in the specified directories.

check_sd_existence()

Check that the FITS files in the singledish key all exist in the specified directories.

get_all_configs()

Get a list of all interf and feather configs.

get_all_mosaic_targets()

Get all mosaic targets defined in “linearmosaic_definitions.txt” which have parts.

get_all_non_mosaic_targets()

Get all targets which are not mosaic targets.

get_all_targets()

Get all targets defined in “target_definitions.txt”, including mosaic targets and their parts and non mosaic targets.

get_alma_download_restrictions(target=None, product=None, config=None)

Get any download restrictions for ALMA data

get_ang_res_dict(config=None, product=None)

Return the angular resolutions for derived product creation for a combination of resolution and spectral product.

get_array_tags_for_config(config=None)

Get the list of array tags associated with an interferometric configuration.

get_channel_width_for_cont_product(product=None)

Get the channel width (in km/s) associated with a continuum product.

get_channel_width_for_line_product(product=None)

Get the channel width (in km/s) associated with a line product.

get_clean_scales_for_config(config=None)

Return the angular scales used for multiscale clean for an interferometric configuration.

get_cleanmask_dir_for_target(target=None, changeto=False)

Return the release working directory given a target name. If changeto is true, then change directory to that location.

get_cleanmask_filename(target=None, product=None)

Get the file name of the clean mask associated with a target and product.

get_continuum_products(only=None, skip=None)

Get a list of continuum ‘products’.

Modified by keywords only, skip. Will only return targets in only, skip targets in skip.

get_contsub_combinespw(product=None)

Query whether the continuum subtraction should combine all SPWs.

get_contsub_excludefreqrange(product=None)

Get the frequency range to force excluding for continuum subtraction for a line product.

get_contsub_fitorder(product=None)

Get the fitorder to be used for continuum subtraction for a line product.

get_contsub_flagedgefraction(product=None)

Get the frequency range to force excluding for continuum subtraction for a line product.

get_derived_dir_for_target(target=None, changeto=False)

Return the derived working directory given a target name. If changeto is true, then change directory to that location.

get_derived_kwargs(config=None, product=None, kwarg_type='strictmask_kw')

Get the dictionary of keyword arguments from the derived key for masking or noise estimation. Valid kwarg_types are ‘strictmask_kw’, ‘broadmask_kw’, ‘noise_kw’ ‘mask_configs’, ‘ang_res’, ‘phys_res’, ‘noise_kw’, ‘strictmask_kw’, ‘broadmask_kw’, ‘convolve_kw’, ‘moments’, ‘shuffle_kw’, ‘flatstrictmask_kw’, ‘flatbroadmask_kw’

get_distance_for_target(target=None)

Get the distance (in Mpc) associated with a target. If the target is part of a mosaic, return the distance to the whole galaxy.

get_feather_config_for_interf_config(interf_config=None)

Get the interferometric configuration to go with a feather configuration.

get_feather_configs(only=None, skip=None)

Get a list of feathered single dish + interferometer array configruations.

Modified by keywords only, skip. Will only return targets in only, skip targets in skip.

get_field_for_input_ms(target=None, project=None, array_tag=None, obsnum=None)

Return the science field given a target, project, array_tag, obsnum combination.

get_file_for_input_ms(target=None, project=None, array_tag=None, obsnum=None)

Return the full file path to of an input measurement set given a target, project, array_tag, obsnum combination.

get_freq_ranges_for_cont_product(product=None)

Get the frequency ranges (in GHz) associated with a continuum product.

get_imaging_dir_for_target(target=None, changeto=False)

Return the imaging working directory given a target name. If changeto is true, then change directory to that location.

get_imaging_recipes(config=None, product=None, stage=None)

Return the imaging recipe for the input config and product.

get_interf_config_for_feather_config(feather_config=None)

Get the feather configuration to go with an interferometric configuration.

get_interf_configs(only=None, skip=None)

Get a list of interferometer array configurations

Modified by keywords only, skip. Will only return targets in only, skip targets in skip.

get_joint_imaging_dirs_for_singledish_config(config='tp')

Get joint_imaging_dirs.

get_joint_imaging_suffix_for_singledish_config(config='tp')

Get joint_imaging_suffix.

get_line_products(only=None, skip=None)

Get a list of line ‘products,’ i.e., line plus velocity resolution combinations.

Modified by keywords only, skip. Will only return targets in only, skip targets in skip.

get_line_tag_for_line_product(product=None)

Get the line tag (in utilsLines) associated with a line data product.

get_lines_to_flag(product=None)

Get the list of lines to flag when either constructing a continuum product or carrying out continuum subtraction on a line product.

get_linked_mask_configs(config=None, product=None)

Return the list of linked configurations used in making hybrid masks.

get_linmos_targets(only=None, skip=None, first=None, last=None)

List all linear mosaics.

Modified by keywords only, skip, first, last. Will only return targets in only, skip targets in skip, and return targets alphabetically after first and before last.

get_moment_list(config=None, product=None)

Return the list of moments to build for a config + product. masks.

get_mosaic_target_for_parts(target_part_name=None)

Get mosaic target name “ngc4321” given a part name like “ngc4321_1”. This is the inverted operation of get_parts_for_linmos.

This is also the same as is_target_in_mosaic(target_part_name, return_target_name=True).

get_overrides(key=None, param=None, default=None)

Check the override dictionary for entries given some key, parameter pair.

get_params_for_moment(moment=None)

Return parameter dictionary for a moment.

get_params_for_singledish(singledish_config=None)

Return parameter dictionary for a singledish config.

get_parts_for_linmos(target=None)

Return the parts for a linear mosaic.

get_path_for_casaversion(casa_version=None)

Get the CASA path for a given version of CASA.

get_phasecenter_for_target(target=None)

Return the strings (ra, dec) that define the phase center for a target in the target dictionary.

get_phys_res_dict(config=None, product=None)

Return the physical resolutions for derived product creation for a combination of resolution and spectral product.

get_postprocess_dir_for_target(target=None, changeto=False)

Return the postprocessing working directory given a target name. If changeto is true, then change directory to that location.

get_release_dir_for_target(target=None, changeto=False)

Return the release working directory given a target name. If changeto is true, then change directory to that location.

get_require_tags_for_config(config=None)

Get the list of required array tags associated with an interferometric configuration.

get_sd_filename(target=None, product=None, nocheck=False)

Return the single dish filename for a target and product combination.

get_singledish_configs(only=None, skip=None)

Get a list of singledish configruations.

Modified by keywords only, skip. Will only return targets in only, skip targets in skip.

get_singledish_dir_for_target(target=None, changeto=False)

Return the release working directory given a target name. If changeto is true, then change directory to that location.

get_statwt_edge_for_line_product(product=None)

Get the velocity width of the edge region to use when running statwt on a line product.

get_system_velocity_and_velocity_width_for_target(target=None, check_parent=False)

Inputs

check_parent: bool, optional

If part of a linear mosaic, return the vsys, vwidth of the parent target.

get_targets(only=None, skip=None, first=None, last=None)

List the full set of targets.

Modified by keywords only, skip, first, last. Will only return targets in only, skip targets in skip, and return targets alphabetically after first and before last.

get_targets_in_ms_key(only=None, skip=None, first=None, last=None)

List all targets that have uv data.

Modified by keywords only, skip, first, last. Will only return targets in only, skip targets in skip, and return targets alphabetically after first and before last.

get_timebin_for_array_tag(array_tag=None)

Get the timebin for an array tag. Returns 0s by default.

get_vfield_dir_for_target(target=None, changeto=False)

Return the vfield working directory given a target name. If changeto is true, then change directory to that location.

get_vfield_file_for_target(target=None)

Get the velocity field file associated with a target. If the target is part of a mosaic, return the velocity field file for the whole galaxy.

get_whole_targets(only=None, skip=None, first=None, last=None)

List only full galaxy names (no parts, e.g., _1 or _2). Very similar to the directory list.

Modified by keywords only, skip, first, last. Will only return targets in only, skip targets in skip, and return targets alphabetically after first and before last.

get_window_for_target(target=None)

Get the velocity window (in km/s) associated with a target. If the target is part of a mosaic, return the velocity window to the whole galaxy.

has_data_for_config(target=None, config=None)

Test whether a target has data for a configuration in the ms key. If “strict” is TRUE then require that a target has data from ALL arrays that make up the configuration.

has_overrides_for_key(key=None)

Check whether the override dictionary contains the input key or not.

The key is usually a file name or a galaxy name.

has_singledish(target=None, product=None)

Return true or false indicating if this target and product combination has associated single dish data.

is_target_in_mosaic(target, return_target_name=False)

Return true or false depending on whether the target is in a linear mosaic.

is_target_linmos(target=None)

Return True or False based on whether the target is a linear mosaic. True means that this target is the OUTPUT of a linear mosaic operation.

loop_over_input_ms(target=None, config=None, project=None, check_linmos=False)

Loop over the target name, project tag, array tag, and obsnum for each input visibility file. If a target is supplied then restrict to that target, trying to match to a linear mosaic if the target is not represented in the dictionary. If a configuration is supplied, then only include array_tags that contribute to that configuration.

Note that the interaction with configs that contain multiple arrays is tricky. Use the ‘requires’ keyword in the config_definitions file to control this. By default, we require ALL arrays that make up the configuration, but this can be changed to an OR if you only need one of a certain combination, e.g.

interf_config 12m {‘array_tags’:[‘12m_1’,’12m_2’]} interf_config 12m {‘requires’:[‘12m_1|12m_2’]}

requires only one of 12m_1 or 12m_2 to be present. If an array is not in the ‘requires’ list, then it is assumed to be required.

make_missing_directories(ms_root=False, imaging=False, postprocess=False, derived=False, release=False)

Make any missing imaging or postprocessing directories.

print_configs()

Print out the configurations for inspection.

print_derived()

Print out the information for derived products.

print_missing_distances()

Print out the missing distances

print_missing_targets()

Print the targets missing a definition in the target list key.

print_missing_window()

Print out the missing velocity window

print_products()

Print out the products for inspection.

set_dochecks(dochecks=True)