Stages of the JWST Science Calibration Pipeline are described in this article. 

The four stages of the JWST Science Calibration Pipeline

The figure below shows the flow of data through the JWST calibration pipeline. This pipeline and the corresponding science data products that it produces can be divided into 4 main stages, depending on the degree of processing:

• Stage 0: Produces uncalibrated raw data products from single exposures in units of total DN (e.g., "uncal.fits")
  
• Stage 1: Produces data products that have been corrected for certain detector effects and converted to units of DN/s (e.g., "rate.fits")
  
• Stage 2: Produces calibrated data products from single or multiple exposures with world coordinates and photometric information (e.g., "cal.fits")
  
• Stage 3: Produces calibrated data products resulting from the combination of multiple exposures into a single integrated product (e.g., "i2d.fits", "s3d.fits", "x1d.fits")
  

Summaries of the algorithms used for individual pipeline stages can be found in the links below. For further information on the JWST science data products themselves, see either the JDox data products overview or the more extensive ReadTheDocs Data Product Types.

See also Key Differences for JWST Time Series Observations and Key Differences for JWST Moving Target Observations.

Stage 0 Pipeline

The Science Data Processing (SDP) system performs pre-processing of the raw telemetry downloaded from the spacecraft prior to the start of the JWST Science Calibration Pipeline. SDP processing is the stage at which key header information is populated from a variety of sources including onboard telemetry, APT proposal inputs, JPL spacecraft ephemerides, etc. The SDP code is distinct from the JWST Science Calibration Pipeline, and general users will not interact with it, instead starting any offline processing with the uncalibrated data (i.e., "*uncal.fits" files) that it produces.

These "*uncal.fits" files are the inputs to stage 1 of the calibration pipeline and usually have 4 dimensions since all JWST detectors use up-the-ramp readout (sometimes referred to as MULTIACCUM) in which pixel values increase between groups within a given integration. The first 2 dimensions are the column and row axes of the detector, while the 3rd dimension is determined by the number of groups per integration, and the 4th dimension by the number of integrations in the exposure. Note that all 4 dimensions will be used, even if N_group = 1 or N_int= 1.

These data come from single exposures and are usually contained within a single FITS file. However, when the raw data volume for an individual exposure is large enough, like for time-series observations, the uncalibrated data can be broken into multiple segments less than 2GB each, so as to keep total file sizes to a reasonable level. Such broken-up exposures usually include "segNNN" in the file names, where NNN is 1-indexed and always includes any leading zeros.

Stage 1 pipeline

Input: Uncalibrated ramp data ("*uncal.fits" files)

Output: Uncalibrated slope images ("*rate.fits" files, "*rateints.fits" files)

Step code overview: 

• calwebb_detector1 (applied for all data)

The first stage of the JWST Science Calibration Pipeline applies detector-level corrections to raw non-destructively read "ramps" from the uncalibrated data in order to produce 2-dimensional count rate images per exposure (or per integration for some modes). These include corrections for dark current, flagging of known bad pixels, countrate non-linearity, jumps in the ramps produced by cosmic rays, and many other effects. This stage consists of a single pipeline that is used for all JWST data; for more information about these steps see the calwebb_detector1 overview article.

Stage 2 pipeline

Input: Uncalibrated slope images ("*rate.fits" files, "*rateints.fits" files)

Output: Calibrated slope images ("*cal.fits") 

Step code overview: 

• calwebb_image2 (applied to all imaging data, including coronagraphy and NIRISS AMI)
• calwebb_spec2 (applied to all spectroscopic data)

The second stage of the JWST Science Calibration Pipeline calibrates the ramps provided by the first stage of the pipeline. Individual steps include pixel flat-fielding, derivation and attachment of world coordinate information mapping detector pixels to wavelengths and sky coordinates, application of spectrophotometric calibration factors, etc. The output of this stage is calibrated data from individual exposures (typically in units of MJy/sr) that is still on the native detector pixel grid. In this stage, processing splits into 2 main workflows, with the calwebb_image2 and calwebb_spec2 pipeline handling imaging and spectroscopic data respectively.

Stage 3 pipeline

Input: Calibrated slope images ("*cal.fits") 

Output: Science-ready mosaics, IFU data cubes, and one-dimensional spectra (e.g., "*i2d.fits" files, "*s3d.fits" files, and "*x1d.fits" files).

Step code overview: 

• calwebb_image3 (applied to MIRI, NIRCam, and NIRISS direct imaging data)
• calwebb_coron3 (applied to MIRI and NIRCam coronagraphic imaging)
• calwebb_ami3 (applied to NIRISS aperture masking interferometry data)
• calwebb_spec3 (applied to MIRI MRS/LRS spectroscopy, NIRCam and NIRISS WFSS, and NIRSpec MOS/FS/IFU spectroscopy)
• calwebb_tso3 (applied to all time-series observations, both photometry and spectroscopy)

The third and final stage of the JWST Science Calibration Pipeline takes the individual calibrated exposures provided by the second stage of the pipeline and combines them into final science-ready data products. For imaging modes, this typically involves combining individual dithered exposures onto a common regularly-sampled mosaic grid, in addition to producing basic source catalog information from this mosaic. For spectroscopic modes, this stage can involve resampling dithered observations into composite 3-dimensional data cubes (for NIRSpec and MIRI integral field spectroscopy) and/or producing final one-dimensional extracted spectra. Data products are typically provided in units of MJy/sr (for images and IFU data cubes) or Jy (for one-dimensional spectra). This stage involves the most unique processing steps for each of the JWST observing modes, as the final science data products can differ substantially between these modes. As such, stage 3 processing uses different pipelines for direct imaging, coronagraphy, aperture-masked imaging, and spectroscopy.  Additionally, observations obtained in time-series (TSO) mode have a different stage 3 pipeline tailored to the needs of the TSO community.