Identifying Potential JWST Duplicate Observations
Investigators preparing JWST proposals can use the MAST Data Discovery Portal or a MAST application program interface (API) to identify potential duplications between targets they wish to observe and previously approved or executed JWST observations.
When preparing a JWST observing proposal, investigators are obliged to check their targets against existing or planned JWST observations for potential duplications. You may use the MAST Data Discovery Portal, or a MAST application program interface (API), to discover these observations of most targets. In addition to the program ID of any existing data, these applications simultaneously provide important ancillary information for the matched observations, such as the instrument(s) and some information about the observing configuration(s). If potential duplications are identified, you will need to inspect the public APT file of the accepted programs to get detailed exposure information to judge whether a duplication is genuine. The portal can also display aperture footprints of archived and planned JWST observations, as well as those from other hosted missions such as HST, which may be useful to plan your observations in detail.
Portal target search
The check for potential duplicate observations begins with searching MAST for your targets, or positions on the sky, with a radius that is appropriate for the instrument configuration you intend to use (see Table 1).
Table 1. Areal Observations—default duplication search radii
General search procedures for observations of astronomical targets are described in Data Exploration with the MAST Portal. Exposures that overlap the selected target (or field) within the search radius will be listed in a table, and the approximate footprints of those exposures will appear against a background image of the sky at the target location.
Potential vs. Actual Duplications
The MAST capabilities described here will help you identify potential duplications between your intended observations and those that are planned or that have already executed. Note, however, that the complete footprint of planned observations may not be accurately represented (see Evaluating Potential Duplications below). You must evaluate the details of the planned observations by using the accepted program's APT file (and/or the Aladin display in APT, as appropriate) to determine if the potential duplications are genuine in the context of the JWST Duplicate Observations Policy.
You may search for fixed targets by entering the target coordinates and a search radius in the Portal (see below). The appropriate search radius to use for duplication checking is summarized in Table 1, and depends upon the instrument and observing configuration. Figure 1 shows a search in a region near 30 Dor from multiple JWST programs.
Words in bold are GUI menus/
panels or data software packages;
bold italics are buttons in GUI
tools or package parameters.
To search 30 Dor for potential duplicating observations, enter the coordinates and search radius: 05:38:42.396 -69:06:03.36
After the search results are displayed, in the Filters panel (far left), find the Mission filter and select the
- Optionally, dismiss the filter panel to see more of the results table and the AstroView panel.
The result will be the footprint of the extant or planned JWST observations for this target (but see the caveat below). While you can edit the columns that appear in the List View table, be sure to display at least the following to see important metadata:
- Exposure Length
- Proposal ID
- Distance (that is, separation of the target from the entered position)
The Proposal ID text in each row is a link to the corresponding program information page, which contains a link to the program summary (.pdf) and the accepted APT program definition (.aptx) file. Alternatively, you may enter the program ID number in the top level APT File pulldown menu to make sure you access the most recent version of the program. You may need to slide the horizontal scroll bar to the right to see the Proposal ID column.
The spatial extent (footprints) for all planned or executed exposures will be displayed in AstroView, superimposed upon a background survey image. Note that the AstroView panel displays all footprints from all search tabs in your Portal session.
If the observations for other missions overlap your intended target, they may prove useful for detailed observation planning (or if from HST, as a pre-image).
Searching for moving targets (mostly Solar System bodies or their satellites) is possible, but rather than entering celestial coordinates (which may not be correct until the observations are scheduled), click the Advanced Search link under the Portal dialog box. When the search panel appears,
JWSTin the Mission filter
1in the Moving Target filter
as shown in Figure 2 below, then click the Search button at the top left of the page.
Some footprint details of moving target or of parallel observations, such as the coordinates or orientations, are subject to change until the observations have actually been scheduled for execution. The AstroView panel is unlikely to be useful in these cases.
Brute force approach
Many moving target names in the planning database are non-standard, so a search on the target name using the Advanced Search interface may not be complete. A simple search for JWST moving targets with no other selection criteria might suffice but also might be overwhelming. For reference, by Cycle 2, about 2,830 observations of moving targets were in the system; this will only increase as additional cycle's observations are included. You might want to write the search results table to a file on your machine for a detailed examination of target names that will help you limit your search.
MAST API target search
As an alternative, you can use the MAST application programming interface (API) to search the archive for planned and archived observations, the mechanics for which are described in the MAST documentation Programmatic Interfaces (see also the examples linked from Using MAST APIs in the JWST Archive Manual). The most popular approach when creating scripts using the API is to use the Python package astroquery.mast. The API allows any search that can be specified with the Portal; the results will be textual rather than visual. This approach is most useful for identifying potential duplications with a large number of targets. You may still want to use the MAST Portal to visualize those results in detail. A Jupyter Notebook with example queries of the database is available.
Download the Duplication Checking notebook from the MAST notebook repository
Use and adapt the examples to your specific needs.
The API is currently superior to the Portal for discovering critical attributes of the observations, such as the
TSOVISIT flag to indicate a timeseries observation. These values can be important for understanding whether instrument configurations and data-taking modes for existing or planned observations are similar to observations you are considering.
Evaluating potential duplications
If the results of one of the above types of MAST Portal searches shows that existing or planned JWST observations coincide with or significantly overlap your specified target/field position, there is a potential duplication. You should go on to the next step, which is to compare the instrument configurations of the existing or planned observations with those you have in mind for your program. Specifically, review the
PUPIL keywords to determine the extent of wavelength overlap, the
TEXPTIME keyword for the total expsoure duration, and also
EXP_TYPE value to understand how the instrument was configured. Finally, look at the accepted program's detailed specification to assess the situation more fully.
The portal does not provide the full metadata about planned observations that are necessary to determine whether your intended observation is a duplication. While the Portal displays footprints for an observation, the exact position for parallel observations cannot be known for planned observations until they are scheduled for execution. Use the process below to visualize the expected full footprint.
Check the APT file
You will need to examine the specifications and exact overlap of planned observing programs in order to determine in detail the nature of a potentially duplicating observation. To do this, fetch the relevant APT file(s) for the specific JWST program(s):
- Click the Proposal ID link in the portal listing (you may need to slide the horizontal scroll bar to the right to see the Proposal ID column). Or if you prefer, use the JWST Program Search Tool) and enter the Proposal ID number to bring up the program planning page for that ID.
- Click the link provided there to download the APT file to your local disk. Optionally, you may download the Observation Summary (Public PDF) file, which is a formatted readable version of the proposal.
- Open APT and load the .aptx file like you would with any other APT file. Or alternatively, open the PDF summary file to view its contents.
- Compare the instrument used and its configuration (imaging/spectroscopy, filters, dispersers, masks) to your intended observation. If the observation is a mosaic, visualize the footprint of dithered or mosaicked observations in APT by clicking the View in Aladin button.
Finally, review the JWST Duplicate Observations Policy carefully to determine if an apparent duplication is genuine.
Resolving genuine duplications
If there is a genuine duplication, there are a few choices for resolving it:
- Select a different target.
- Change your observation in a way that does not duplicate those already planned or executed.
- Make note of the duplication and include a sound justification for it (e.g., observations of a time variable source) in the appropriate section of your proposal's Scientific Justification, which is part of the PDF file that must be attached to your APT Proposal Information page prior to final submission of your proposal.
Note: all observations obtained in DD/ERO and DD/ERS programs, and observations in select GO and GTO programs, are publicly available as soon as they have been archived, with zero exclusive access period. Archival proposals for these data are part of the annual proposal solicitation. See the Call for Proposals for details.
Resolving potential duplications
In certain cases, you will only be able to identify that there are potential duplications with your proposed observations. The most likely case where this will happen is for NIRSpec MOS observations where the detailed target selection is not made until after proposal acceptance. Also, accepted NIRSpec MOS programs for which detailed target selection has not taken place will also only produce potential duplications at the time you are proposing. In these cases, simply discuss the potential for duplication in your proposal and the detailed duplication checking will be done at a later date after proposal acceptance.