NIRSpec MPT - Catalogs

Requirements for creating a Catalog of sources, and ingesting it into the MSA Planning Tool are presented in this article. Candidate sets are also introduced. 

On this page

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

The MSA Planning Tool (MPT) is used for planning MSA Configurations to observe a set of sources distributed across the MSA aperture projected on the sky.  The tool requires the ingest of a catalog of source centers in order to run. Even for proposers waiting for NIRCam or other pre-imaging for accurate source positions, an input catalog is needed. Users waiting for pre-imaging should create a fake catalog with similar areal coverage and density to design their MOS proposals in MPT.

Optimal MOS spectral calibration for aperture flux loss and wavelength solutions will require Catalog measurements with high-precision relative astrometry from an existing image. The MOS mode observations can be planned using astrometry with accuracies from 5–50 mas. High precision spectral calibration needs relative positional accuracy of 15 milliarcsec or less, which typically requires space-based observations. Images obtained using HST/ACS or HST/WFC3 UVIS within the past 10 years should have this level of accuracy. In some cases, the planning of the NIRSpec spectroscopy will require preceeding NIRCam observations, a process that is called NIRCam pre-imaging. Nonetheless, while high-precision relative astrometry is highly recommended, it is not a requirement.

Cautionary advice concerning MOS Catalogs during program development

CAUTION:  At any point during MOS program development, do not delete a Catalog if you have plans in MPT, or MOS observations in the program, that were made using the Catalog or any of its Candidate Sets. This applies to both executed and un-executed observations. When you are ready to improve your catalog from pre-imaging observations, you should be careful to replace the existing catalog with a new catalog of the same name.

During program development, it is also recommended to keep all MPT Plans that were used to create MOS observations in your program. This makes it easier for reviewers or help desk staff to track down any problems.

Once an angle has been assigned to your MOS observation, please do not delete the observation or its plans, or the angle assignment will be lost.

After the MOS program updates are finalized, (i.e., the old MOS observation has been updated from the re-planned MPT plans and the updated Catalog) you may clean up if you wish. First, delete old MPT plans that are no longer used in any observations, then delete the old Catalog and Candidate sets used to make the old plans.  Again, make sure to keep the updated Catalog and the MPT plans that went into the creation of the finalized MOS observation(s). Contact the JWST Help Desk if you are uncertain.



Creating a source Catalog

See Also: NIRSpec MOS and MSATA Observing Process, MPT Catalogs - Examples, NIRSpec MSA Target Acquisition, NIRSPEC MPT - MOS and MSATA Program Updates

Parent source Catalogs can have as many objects as needed. It is recommended that Catalogs include all known sources in the field in order for the user to spot contaminants and prevent the MPT from placing them into failed-open shutters. The MPT Catalogs - Examples article presents several examples to guide the user on Catalog formatting. Tables can be delineated by whitespaces, tabs, or commas. Virtual Observatory tables are also supported. For flight-ready program updates (but not proposals), the Catalog must also include reference stars for MSA target acquisition. Some useful information about updating the source Catalog for the MOS program update submission are presented in the next section.

Parent source Catalogs must be created with these simple rules in mind:

  • Two columns are required for the equatorial celestial coordinates (J2000) RA and Dec for each source. These can be expressed in units of degrees or in hexadecimal format. 
  • It is useful to associate each source position with an integer ID in the first column of the Catalog. The ID should be an integer value from 1 up to 1e6 (1000000). The ID will be encoded into the name of the Level 3 output products from the pipeline.
  • It may be useful for at least one column to have source fluxes or magnitudes (optional).
  • Optionally include a column called Stellarity with a value from 0 to 1 for increasingly point-like sources. Use a value of -1 for unknown source stellarity. This will be used by the pipeline for processing the sources. Default processing is for point-like objects.
  • If a column of source Weights is present, it is important to note that the weight value should not exceed about 1 million (1,000,000). This limit is due to the fact that, currently, the sum of all source weights in a Plan in MPT is computed as an int sum.
  • The Catalog may optionally contain a header line at the top of the file indicating column names separated by whitespace. The header line must begin with a "#".  
  • Using recognized labels will assist ingest and filtering of the Catalog when making subset lists called Candidate Sets. Some recognized labels are ID, RA, DEC, Size, Redshift, Reference, Stellarity, Magnitude, Magnitude error, R50, Number, Label, Weight, and FWHM. Most of these are optional columns.

There are some checks that should be done before ingesting your source Catalog into MPT. These are key to avoiding problems with MPT planning and downstream data processing:

  • Commas should not be present in the source Catalog, unless they delineate columns in a CSV table. They are not handled correctly when ingesting the Catalog to MPT.
  • There should be no duplicate entries in your source IDs. Different source positions should have a unique ID.
  • There should be no NULL (or empty) entries in the Catalog. MPT may read the Catalog and assign a value of zero to such data. This would be particularly problematic for a magnitude column and could result in incorrect results in the derived candidate sets made from the Catalog. Instead, choose a value (real or integer) that can be easily filtered out.



Updating a source Catalog with candidate reference stars for the MOS program update

See Also: NIRSpec MPT - MOS and MSATA Program Updates, NIRSpec MOS and MSATA Observing Process

Observers will generally need to update their source Catalog before submitting the initial MOS program update. This is especially true when NIRCam pre-imaging has been requested. In that case, actual source positions may have improved, and there will likely be new sources to add to the Catalog. Even for programs without NIRCam pre-imaging observations for which no changes are needed to sources or their positions, the Catalog will still need to be updated to mark suitable candidates for reference stars if using MSATA. A more detailed discussion of Catalog updates for preparing the MOS program update can be  found in the article on NIRSpec MPT - MOS and MSATA Program Updates.



Ingesting a source Catalog

How to ingest the source Catalog in APT

To start, the Catalog should be ingested into APT from the Targets folder (Make sure you are in the Form Editor). The button "Import MSA Source Catalog", highlighted in Figure 1, can be used to create an MSA Catalog Target.

Figure 1. Importing a Catalog of sources

A pop-up window will then appear for loading the Catalog called the MSA Source Importer, shown in Figure 2. Several formats are allowed for the Catalog: CSV - Comma-Separated Values, TSV - Tab-Separated Values, Whitespace Separated, or VOT - Virtual Observatory Table format. Select the appropriate format under File Format. Give the Catalog a Catalog Name. The name should be from 2–31 characters, and only letters, numbers and "+", or "-" are allowed. Underscores are not allowed. The name should not have spaces. 
Figure 2. Selecting a File Format


Defining the working set of columns

If the File Format has been selected properly in the MSA Source Importer pop-up window, MPT will attempt to auto-identify or label the columns (Figure 3) with the help of the header. Users should correct the labels using the pull-down menu located below each column. A number of common options are available: ID, RA, DecMagnitude, Magnitude ErrorRedshiftFWHM, R50, Weight, Reference, Stellarity, Number, and Label. Unrecognized columns will have the default label Ignore. All columns with the label Ignore will not be read into MPT. If the Catalog has many columns that you don't need for planning, then it is convenient to leave them labelled as Ignore. If you plan to use a column in MPT for making a filtered Candidate List, and it does not appear in the list shown, select Number if it contains numerical data, or Label if it has alpha-numeric values.

Figure 3. Assigning a type to each column


This MSA Source Importer window pops up when loading a Catalog as an MSA Catalog Target. The window is expandable. MPT will try to identify the data type in each column, but it may need assistance in doing so. The user should help to identify the data type using the pull-down menu below each column. Only the columns you wish to have displayed in APT should be identified in this way.

Important caveats

Despite all the nice options listed above, only columns labeled Magnitude, Size, Redshift, Weight, or Number can be used to create Candidate Sets or Candidate Lists in APT, by filtering the parent Catalog in APT. Since Candidate Lists will normally be used to create MPT Plans, it is important to ingest the data carefully at the start, so the essential columns become available for later filtering.

There are some known issues regarding catalog ingest and filtering:

First, if you plan to use Candidate Lists (subsets of the parent Catalog) for planning in MPT, make sure to ingest those columns as one of these 4 types: Magnitude, Size, Redshift, or Weight. If none of those options make sense, select the Number option. The "Number" type has been provided as a work-around. A detailed example is shown in an expandable panel below this box.

Alternatively, or if the workaround described here fails, identify the subset of sources for your candidate set outside of APT, and add a new column to the Catalog containing a unique numerical value to mark this set of sources. Ingest the column as a Number column so you can work with it in APT.

Secondly, one or a few of the recognized column types listed in the first paragraph of this subsection may not be displayed after catalog ingest. Magnitude Error has this problem. If you want to use this column for filtering, then, once again, select the Number label during ingest. The column will then be displayed, and the data should then become available for filtering (using the Number filter) to create a Candidate List.

An example showing a step-by-step workaround for creating a Candidate List using the Number column type

Let's say your Catalog contains a FWHM column, and you wish to use it to filter the Catalog to create a Primary Candidate List for use with MPT. The trick is to load it into APT in the following steps:

  1. Load the Catalog into APT as shown in Figures 1 and 2 above. By default, the FWHM column will be indicated as FWHM.

  2. Change the label below the column to Number by highlighting that option in the pull-down.

  3. Click Import to ingest the Catalog, and then specify the Astrometric Accuracy and Pre-Image Availability.

  4. Note that the Catalog data displayed does include the FWHM column data (the last column in the figure above), but it is called FWHM, not Number. Column types read in as Number columns will not be displayed at all, unless they are one of the 13 or so types listed above Figure 3 that are automatically recognized by APT.

  5. Now, we want to create a Primary Candidate List for observation planning which contains only those sources within a specific range of values of FWHM. Click the "New Candidate Set" button below the highlighted Catalog (See previous figure). In the pop-up window (shown below), click the "Add Filter" button, and select the New Number Column option from the pull-down.

  6. In the new smaller pop-up window, pull down the Column menu and select FWHM. Enter a Minimum and/or Maximum FWHM value for your Candidate Set.

  7. Click OK, and Make Candidate Set. The new Candidate Set will then be displayed. Note that it contains only a subset of the parent Catalog sources.

Once the Catalog is ingested, selecting or highlighting it will display the data.  Figure 4 shows how the sources in a Catalog are displayed when the Catalog is highlighted within the main window of the GUI.

Scrollbars are present to help navigate the Catalog. The Catalog may be sorted (in display mode only) by clicking on any column name. Columns may be reordered by dragging and dropping. The Catalog will maintain its initial ordering for making an observation plan.

Figure 4. A Catalog successfully loaded to the MSA Planning Tool

A parent Catalog as displayed in MPT. Note the scroll bars. Scroll bars are available to help users navigate the Catalog. Reorder columns by dragging and dropping. Note that the re-ordered Catalog displayed here if for convenience only, it has no effect on Catalog ordering as used in MPT to populate the MSA Configurations, for example. A Comments tab is provided. It may be useful to comment on the magnitude units in the Catalog, for instance.


Catalog properties

Source ID

The source ID is an optional column. The ID is read as an int. Values between 1 and 1e6 are acceptable. Plan weights are also computed as ints, so to allow for sums of a reasonably large number of sources in a Plan, the upper weight limit for an individual source should be kept below 1e6.

Target weights

Weights may optionally be used to create the plans. In the absence of source weights, MPT will prioritize sources based on the ordering of the sources in the Primary candidate set (which is derived from their order in the Catalog), described below. If some sources in your Catalog have a higher priority than others, then it is advisable to create an additional column called Weight in the Catalog and assign higher values for the weights to those sources. When this Catalog is ingested, assign the weight column as a Weight column type. If the use of source weights is enabled in the Planner, sources with larger weights are more likely to be observed. If the Catalog does not have a weight column, MPT will assign the weight 1 to all sources.

Source weights should be kept under a value of 1e6. They are read in as integers, and the computation of the total plan weight is performed as integer math. A total plan weight is the sum of all observed sources, and exceeding the upper limit for integer math can cause an improper selection of the best pointing by MPT, and hence a less optimally efficient plan.

Note that if the user elects the Use Weights option in the MPT Planner, MPT will use the weights in the Catalog in a linear way. E.g., a source with a weight of 100 will count as much as 100 sources with a weight of 1 when the tool is attempting to optimize the MSA Configuration.

Astrometric accuracy

A catalog relative Astrometric Accuracy in units of milliarcseconds must be specified after Catalog ingest. If a Catalog comes from, for example, an HST/WFC3/UVIS observation, one may assign the value 12–15 milliarcseconds. This value is used to estimate the final accuracy of the target acquisition when using the MSA target acquisition (MSATA) method.

Important Recommendation - Register catalog to GAIA DR3

Imaging data used to derive the astrometry for your catalog of reference stars and science targets need to be carefully vetted against Gaia stars. In particular, the MSA target acquisition procedure has a very limited ability to correct for errors in the initial roll, and so it is strongly recommended that the rotation of the reference stars on the sky be aligned with that of the Gaia frame to better than one arcminute (60", 0.0167°) in rotation. Roll offsets larger than 500" (0.14°) are likely to fail. Care should be taken to consider any proper motion of the Gaia stars and reference stars to avoid introducing a spurious roll offset larger than this. This is required to produce the best astrometry possible, and to correct for any lingering small roll offsets that may affect target placements.


Catalog Reference Position

The Catalog Reference Position is the reference point for the MOS observations that will use this Catalog. This position is automatically calculated by APT as a median of the Catalog source RA and Dec values, and is not editable.

This point serves as the reference point for the assigned Aperture PA. At proposal submission, the exact pointings within a MOS observation are not yet defined, so the assigned Aperture PA used in scheduling MOS observations is anchored to the Catalog Reference Position. Consequentially, since the observatory roll must be kept constant during an observation, MPT will adjust the Aperture PA (and V3PA) slightly from its value at the Catalog Reference Position for each offset pointing in the observation. This is important to keep in mind for users ingesting MSA configurations created outside of MPT.

Pre-image Availability

NIRSpec MSA observations will require the most accurate relative astrometry in order to place sources in the micro shutters. For this reason, the MPT asks for the Pre-image Availability which has the options and consequences listed in Table 1. This mechanism is used as a way to alert the ground system that there may be an associated observation that will need to be linked to the MOS observations for planning and scheduling. If the option Will be done in this program is selected, APT will ask the user to select the appropriate NIRCam observation in the program. The selection Is already obtained  indicates that there is an existing image in the archive or elsewhere that is associated with the source positions in the Catalog. The reference image in that case will be uploaded by the user with the program update submission and stored in the archive for later archival observers. This reference image is a snapshot of the imaging used at the time of planning the MOS observation, and will correspond to the positions used to create the pipeline source products.


Table 1. Description of the pre-imaging availability options

 Pre-image Availability OptionsDescriptionConsequences
Is already obtained

The image was already obtained using NIRCam, or another imaging instrument, e.g., HST ACS.

With this selection, after proposals are approved, the observer will be expected to upload the (fits format) Pre-Image file to the archive during MOS program update submission. The selection can be changed at that time if it the user prefers not to share the imaging data with future archival users. 

Will be obtained external to this program

The image is not part of this proposal. For example, it will be obtained with an upcoming HST program, or an upcoming NIRCam observation in another program. The imaging will be used to update source positions in the Catalog at the time of MOS program update submission.

The observer will be asked, but is not required, to upload the recent imaging to the archive during MOS program update submission.

Will be done in this programThe user proposes to observe the same field with NIRCam. In this case, 2 options are offered: selecting a NIRCam pre-imaging observation from the same proposal, or selecting a NIRCam coordinated parallel image attached to the program. This option corresponds to the more narrow traditional definition of "pre-imaging" discussed in the article NIRSpec MOS Operations - Pre-Imaging Using NIRCam.These data will be automatically archived for future archival observers.
Not required

In rare cases, an image may not be available and isn't required for the science. For example, precise pointing may not be needed for observing extended sources.

Use this option if you just want to experiment with the MSA Planning Tool (MPT) without getting warnings.

Pre-image upload options during program development

Observers are asked to provide any pre-imaging they have used (or will use) to derive Catalog positions for upload in APT at the time of MOS program update submission. Observers can indicate their intent to provide pre-imaging when it is available during proposal submission. For those MOS programs that have selected the option Is already available (see Table 1), observers will be expected to upload the imaging to MAST via APT so that it can be associated with the derived observed MOS sources in MAST. Associated imaging will be extremely useful for future archival researchers, and may also serve to streamline observation planning for the observing teams. The pre-image, once uploaded, is proprietary until the end of the proprietary period of the program, normally one year. The pre-image, once uploaded, can only be accessed by the PI or their designee during the proprietary period.

APT will present different options at each stage of proposal development.  

(1) In the initial proposal submission, the user ingests an MSA Catalog into the Targets folder in APT.  During the ingest, the observer is asked to select an option for any pre-imaging availability associated with the Catalog (See Table 1 above).  Figure 4 above shows what it looks like in APT prior to program acceptance.

(2) Once the proposal moves to approved status, observers will see a new button on the MSA Catalog Target for selection of the pre-image FITS file on disk, with a red X to indicate the file now needs to be uploaded. See Figure 5.  However, the option to provide imaging, or not, can be changed each time the program is submitted up until execution. Providing a snapshot in time of the imaging used to produce the Catalog positions will be useful for researchers to associate with the pipeline-produced spectral products from the program, which will be labeled by target position in the Catalog.  The upload requires a fits format image, and will be checked for a minimum set of header keywords.

(3) Finally, after the pre-image is attached to the program, APT will display some new buttons on the MSA Catalog Target: Copy to Disk, Replace, Send to Aladin, and Remove from Proposal (see Figure 6). These buttons allow one to copy the image to disk, replace the FITS image with a newer one for the archive, send the image to Aladin for viewing in APT, or to remove the pre-image from the program.

Figure 5. Providing the pre-image at program update submission

A parent Catalog as displayed in MPT, which, when highlighted, indicates that the user should upload the imaging associated with the MSA Catalog when submitting the program update.
Figure 6. New options are available after pre-imaging is uploaded and associated with the program

A parent Catalog as displayed in MPT for an approved program, for which pre-imaging has already been uploaded. When the Catalog is highlighted, as shown, 4 new buttons appear that allow the user to download the imaging, replace it with another image, display it in Aladin, or remove it from the program.

NIRCam pre-imaging that is part of the same program as the MOS observations will not need to be uploaded. The NIRCam images will automatically be saved in MAST, so do not need to be uploaded with the MOS program update.

If at any time prior to observation the observer would prefer not to provide the pre-imaging data, they may select a different pre-imaging option on the MSA Catalog Target, including not providing anything, then they may simply select the ‘Not required’ option.



Candidate sets (Also called Candidate Lists)

Candidate sets are extracted from the supplied source Catalog after it is loaded into MPT. Filters are applied to choose a subset of the source list based on a magnitude range, a type of source (extended vs. point-like), or a redshift range, among other possibilities. Several Candidate sets can be defined from the parent source Catalog.

Observers should be aware that they need to have all sources of interest in a single parent Catalog for a given observation or a set of observations. The candidate lists (Primary Candidate List and Filler Candidate List) are derived from this single Catalog. MPT will prevent the creation of an observation from Plans that were made using different ingested Catalogs.

By clicking on the New Candidate Set button (Figure 6) a new pop-up window is displayed (Figure 7). First insert a Name and begin source filtering by clicking on Add Filter. This button will present a limited set of options for filtering sources from the Catalog: Magnitude, Size, Redshift, Weight, and Number Column. Define the appropriate filter limits for the candidate set, and click OK to close this window. Note that the ranges are always inclusive of the numbers entered. Finally, click "Make Candidate Set".

MPT's calculations are complex and involve multiple iterations. Large source lists can result in long runtimes. The computations have been parallelized to increase efficiency, but the runtime will depend on your computer. (See the MPT Computational Performance article). It is advisable to experiment with smaller Candidate sets to see how long it takes and then add to them as needed.

Figure 7. Creating a Candidate Set by filtering Catalog sources

Creating a candidate set by selecting sources with redshift greater than 3.
Figure 7 demonstrates how to create a Candidate Set (also known as  a Candidate List) of sources from a given Catalog. In the example shown, sources with redshift between 2 limiting values are selected as members of the Candidate SetBy clicking on the Make Candidate Set button, the new Candidate Set is created. As many Candidate Sets as needed may be created using this procedure. 

Source Catalogs and Candidate Sets may be deleted using the Delete button seen in Figures 4-6. Note that a Catalog or Candidate Set used in an observation cannot be deleted until the observation using it is deleted.

A Catalog and/or one of the candidate sets derived from it, can be highlighted in the Targets folder of the GUI and displayed in Aladin by clicking the Send to Aladin button (shown in Figure 4). Figure 8 shows an Aladin view of the spatial distribution of sources from two Candidate Sets. We refer the user to the Aladin documentation for further information about Aladin features.

Having many planes in the Aladin stack may cause APT to run slowly, even after the popup is closed. To clear out all the layers, users should click Edit > Delete All in Aladin.

Figure 8. Example of an Aladin view of two candidate sets
Once the parent source Catalog has been uploaded and the Candidate Sets have been created, the Planner can be used to specify a plan, as explained in the article NIRSpec MPT - Planner.



References

Karakla, D. et al. 2014, Proc. SPIE 9149
The NIRSpec MSA Planning Tool for multi-object spectroscopy with JWST

Aladin 




Latest updates
  •  
    Corrected the advice to users about registering to GAIA and NIRCam pre-images

  •  
    Reviewed and edited to give advice about target ID values, and to emphasize limits on values for stellarity, target ID, and Weight.

  •  
    Catalog update instructions moved to new article MOS and MSATA Program Updates.

  •  
    Clarified target weights.

  •  
    Added a section to describe the addition of columns marking candidate reference stars for MOS program updates.

  •  
    Described how the catalog reference position is computed and how it's used

  •  
    Clarified use of fake catalog for those waiting for pre-imaging.

  •  
    Described important caveats for catalog data ingest into MPT.  Provided a workaround to make it possible to filter on certain columns to make candidate sets for MPT planning.

  •  
    Fixed quotes and APT parameter fonts
Originally published