Running the JWST Science Calibration Pipeline

Standard calibration pipeline processing should produce publication-quality data products but some science cases will require non-default settings in pipeline processing.  

See also: Algorithm Documentation, Stages of JWST Data Processing, Understanding JWST Data Files, JWST Data Associations
ReadTheDocs (External Links): Software DocumentationRunning the PipelineFile Naming Conventions, Data Product Types, Science Product Structures and Extensions, Data File Associations

Standard calibration pipeline processing should produce publication-quality data products. However, your science cases may require specialized processing with settings other than the defaults used in pipeline processing.  Also, while bulk reprocessing will be performed by STScI as conditions and resources permit, you may wish to expedite reprocessing of data sets of interest to you when new calibration and reference files become available.  

Early data will be calibrated using ground-based calibration files; over time, these will be replaced with calibrations based on on-orbit data. Other conditions under which an observer may need to reprocess data will not be known until JWST has undergone commissioning activities and the observatory is better characterized. Once the observatory is returning flight data, STScI will provide guidance for users on whether they should reprocess their data.

Science calibration pipeline stages

There are 3 main calibration pipeline stages required to completely process a set of exposures for a given observation:

  • Stage 1: Apply detector-level corrections to the raw data for individual exposures and produce count rate (slope) images from the "ramps" of non-destructive readouts
  • Stage 2: Apply physical corrections (e.g., slit loss) and calibrations (e.g., absolute fluxes and wavelengths) to individual exposures
  • Stage 3: Combine the fully calibrated data from multiple exposures

Words in bold italics are buttons 
or parameters in GUI tools. Bold 
style represents GUI menus/
panels & data software packages.

Summaries of the algorithms used for the corrections and calibrations can be found in JDox, while more detailed information is contained in the software documentation. Each stage may use different modules depending on the observation mode and instrument. Individual steps and pipeline modules can be run in the following ways:

There are generally 2 types of input: science data files or associations, and reference files. The reference files are provided by the Calibration Reference Data System (CRDS) unless they are explicitly overwritten.

Examples

See also (External Links): JWebbinars

While the pipeline software documentation offers a general description on how to run the pipeline, a number of intricacies exist in the way in which the various software and data products interact. Several Jupyter notebooks have been developed to help you understand your data or to highlight general science workflows that you may want to consider while reducing your own data.




Latest updates
Originally published