JWST Data Reduction Pipeline
The JWST Data Reduction Pipeline is a Python software suite that automatically processes data taken by the JWST instruments NIRCam, NIRSpec, NIRISS, MIRI, and FGS to remove instrumental signatures from the observations.
The JWST Data Reduction Pipeline (typically referred to as the "calibration pipeline") processes data from all instruments and observing modes, producing both fully calibrated individual exposures and high level data products (mosaics, extracted spectra, etc.). It is organized in stages by type of observations. The algorithms and overall structure of the pipeline are decided by an inclusive working group. The details about how to install and run the calibration pipeline are provided.
These JDox pages describe the baseline version of the calibration pipeline as envisioned for operations, at a relatively high level of detail. For more in-depth information on the stages of processing for the calibration pipeline, please see the official software documentation.
Installing the calibration pipeline
Linux and MacOS platforms are tested and supported. Windows is not currently supported.
JWST calibration pipeline software is currently under development. It is not yet supported for public use.
When the software is publicly released, it will be available in AstroConda, which is a free Conda channel maintained by the Space Telescope Science Institute (STScI) in Baltimore, Maryland. Until then, it can be obtained from the GitHub repository in which it is maintained. Please note that this software is still under active development; it is offered on an as-is basis with no guarantee of correctness or support.
Running the calibration pipeline
There are 3 main calibration pipeline stages required to completely process a set of exposures for a given observation:
- Stage 1: Apply detector-level corrections to the raw data for individual exposures and produce count rate (slope) images from the "ramps" of non-destructive readouts
- Stage 2: Apply physical corrections (e.g., slit loss) and calibrations (e.g., absolute fluxes and wavelengths) to individual exposures
- Stage 3: Combine the fully calibrated data from multiple exposures
The algorithms used for the corrections and calibrations are decided by a cross-instrument group, the details of which are also provided in the software documentation for each step. Each stage may use different modules depending on the observation mode and instrument. Individual steps and pipeline modules can be run in the following ways:
There are generally two types of input to a pipeline or steps: science data files or associations, and reference files, which are provided by the Calibration Reference Data System (CRDS) unless they are explicitly overwritten. The software documentation provides the most up-to-date information about parameters, inputs, and outputs for the various stages of processing.
Swade, D., Greenfield, P., Jedrzejewski, R., and Valenti, J., 2010, JWST-STScI-002111
DMS Level 1 and 2 Data Product Design