Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Content Block
overflowauto
meta-propertiestrue
nameSummary
id277319516
classpdf-full

The MAST Discovery Portal is the primary web interface for discovering, visualizing, assessing, and retrieving calibrated data products from the JWST calibration pipeline, as well as calibration and engineering data products and services.


Content Block
overflowauto
nameContent
id277805920
classpdf-full

Introduction

There are a variety of ways for researchers to access JWST data of interest, including the MAST Discovery Portal which is the primary web interface for discovering, visualizing, assessing, and retrieving archived data. The portal also provides access to ancillary and engineering data related to the observations, and data quality summaries that enable users to assess any anomalies during the observations, the achieved exposure depth, and the calibration processing. Users may evaluate contemporaneous calibration reference data that were used to remove the instrumental signature from the science data products. They may also discover data through a programmatic (software-based) interface such as the MAST Applications Programming Interface (API), and through various community tools. Users may elect to be notified as data from new JWST observations become available through MAST.

Data product types

The JWST Data Management System (DMS) produces many products for each JWST observation, including the science files produced by the data reduction pipeline. The exact type and number of products depends upon the instrument, its configuration, and operating mode. Consult the Understanding Data Files article for a detailed description of each science product and the concomitant data. Most of the science data files are images or tables in FITS format (Pence, et al. 2010), while others are in structured or unstructured ASCII format. Table 1 contains a short summary of the data product types that may be included with each dataset. Table 1 below summarizes the semantic content of the various data products, including some that are produced outside the science data reduction pipeline.

 

Table 1. Summary of data product types that may be included with each dataset

Product typeFormatDescription
Science dataFITS MEF imagesAll levels of science data products from imaging, spectroscopy, and time-series observations. The image extensions contain multi-dimensional science pixel arrays, concomitant data quality flags, and variance arrays.
FITS BINTABLEsExtracted spectra are stored as tables, and include fields for spectral coordinates, flux, and concomitant quality information.
FITS FOREIGNMetadata for images or spectra are stored in ASDF format, and are packaged in a FITS extension of type FOREIGN.
Source catalog

FITS binary table

Measurements of detected source properties, derived from image exposures.
AssociationsJSONEnumeration of relationships between all levels of science data products, and related calibration reference products, that were used during the course of calibration pipeline processing.

Association summary report

HTMLA summary of associated exposure and observation attributes, graphics, plots, and completion status.
OPDFITSOptical Path Difference images that describe the state of the mirror segment alignments for a particular epoch. The OPD files can be used with a tool such as WebbPsf to determine the estimated PSFs for any given instrument configuration.

Anchor
MRDP
MRDP
Minimum recommended data products

Of the many different data products produced by the calibration pipeline, a subset has been identified as essential for extracting the intended science from the data. These are termed the "minimum recommended data products" (MRDP). The selection of data products that are included in this set depends upon the instrument used to obtain the data, and its configuration and operating mode. Generally, products in the MRDP include the lowest-level science product for which the instrumental signature has been removed along with the associated data quality products, but exclude calibration and certain ancillary products.

Comment

(Note: The complete list of MRDP per instrument, has not yet been defined.)


Tip

If close examination of the MRDP or the data quality flags in the report indicates problems, it may be wise to retrieve the entire dataset for review and (possibly) custom processing.

Observing program assessment

Observing programs for JWST can be highly complex, consisting of multiple exposures, or observations of multiple targets or fields, some of which are related in the data processing through associations to produce combined or stacked products. The exposures may consist of, e.g.:  

  • Dithered exposures
  • Tiled imaging over an area of sky, in one or more bands
  • Coronographic imaging, including a PSF reference star

Details of the exposures, their relationship to one another, their context in the overall observing program, and the status of their completion for active programs) are summarized in the JWST Associations Summary Report.

Comment

(Note: The Associations Summary Report format and retrieval method will likely evolve prior to launch, necessitating updates to this sub-section.)


Data reprocessing

Data from JWST are likely to be reprocessed with the calibration pipeline from time to time, particularly early in the mission lifetime. Users may also wish to reprocess JWST data for many of the same reasons, including:

  • Additional observations of the target/field become available
  • Better or more appropriate contemporaneous calibration reference files are created
  • New software or calibrations are implemented to remove unexpected instrumental signatures
  • Custom processing is required to achieve the optimal scientific result

The provenance metadata in the file headers will indicate the date, software version, and (unique) calibration file identifiers. In addition, release notes for the pipeline software are published, so that the epoch of processing by STScI will be apparent.

 


MAST Discovery Portal

MAST implements various protocols of the Virtual Observatory (VO) including those for image, table, and spectral data access, and the Common Archive Observation Model (CAOM) that describes the core services offered by MAST. These are the mechanisms by which MAST data can be searched and retrieved by VO-aware applications, and conversely, the mechanisms for the Discovery Portal application to discover, retrieve, and operate on data not hosted in MAST.

Archived data

A portal has been developed for MAST data collections, including JWST. The MAST Discovery Portal offers great flexibility in customizing queries to identify datasets to explore and retrieve. While the interface is straightforward, a user guide may help first-time users. See the Data Exploration article for a detailed example of a search using the Discovery Portal.

Anticipated data

At any given time some observations in a program may have been executed, archived, and become available to the community; some archived observations may temporarily be restricted to those with exclusive access; while still other observations may remain to be obtained. Investigating teams and the broader community each have an interest in data availability. In order to encourage the greatest possible use of JWST data in MAST, a subscription service will notify registered and subscribed users when one of the following observation-related events occur:

  • new observations are Archived,
  • archived data have been reprocessed, or
  • restricted-access data become available to the public.

Users may tune the notifications by mission, program ID, event type, and science product. Users may establish or cancel their subscription through the MAST Discovery Portal, change the media and frequency of notifications, and change the selection criteria for notifications.

Duplicate Observation Identification

MAST provides the capability to check one or more user targets against extant and planned observations to identify potentially duplicate observations. Duplicated observations are, in general, not allowed so it is in the proposer's best interest to perform this check prior to submitting a proposal for review. Read the article Identifying Potential Duplicate Observations to see how to query planned observations.



Engineering data

The JWST engineering database service provides an archival repository of engineering telemetry, including temperatures and other time-dependent status information, about the science instruments and the spacecraft itself. Telemetry associated with specific measurements vs. time are tagged by mnemonics which, in practice, are rather opaque to non-specialist users. Of the roughly 23,000 archived mnemonics, nearly 15,550 of them can be queried, visualized, and downloaded by general users through a Portal interface. Users may access the engineering database via a direct link to the query interface or, after querying the Portal for science data, through a link on each row of a search results table. See the JWST Engineering Database Browser for details.

 

Comment

Wavefront sensing database

(Note: a link to/description of the wavefront sensing database once it exists.)



Virtual observatory tools

Many community software applications are capable of accessing remote data using protocols developed for the Virtual Observatory. You may have used them before without being fully aware of how such data were obtained. Table 2 provides an incomplete list of community, VO-aware applications for visualizing and exploring archived astronomical images, spectra, and catalogs:

Table 2. An incomplete list of community, VO-aware applications for visualizing and exploring archived astronomical data

Excerpt Include
_Inclusions Library
_Inclusions Library
nopaneltrue

 


 



Content Block
overflowauto
nameReferences
id281944791
classpdf-full

References

Pence, W. D., Chiappetti, L., Page, et al. 2010, A&A, 524, A42
Definition of the Flexible Image Transport System (FITS), version 3.0

MAST Discovery Portal

JWST technical documents





Content Block
overflowauto
nameIcon
id278719485
classpdf-hide

Multiexcerpt include
MultiExcerptNameObtaining JWST Data
PageWithExcerptMR:DL Obtaining JWST Data


Content Block
nameUpdates
id1074221578
classpdf-hide


HTML Wrap
padding0
margin0
idupdatesbox


HTML Wrap
tagspan
classupdatesbox-heading

Last updated

Updated March 12, 2018

  • Additional information on identifying duplicate observations, including a link to the "how-to" page

  • Rewritten subsection on the Engineering Database interface, with a link to Data Exploration article


Published July 11, 2017


 

Comment

The format for 'Last updated' is shown below. Enter updates above this internal"Comment" box. Only enter major updates (not typos or formatting changes).


Updated April 5, 2017

  • Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam fermentum vestibulum est. Cras rhoncus. 
  • Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Sed quis tortor. 

Published March 2, 2017