JWST Operational Pipeline Build Information

Information about current and previous build deliveries of the calibration pipeline software, as well as known issues or shortcomings of each build and planned enhancements, are provided in this article.

On this page

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

The JWST Science Calibration Pipeline used by the Mikulski Archive for Space Telescopes (MAST) to calibrate all JWST data is a subsystem of the JWST Operational Pipeline. It is updated on a regular basis. These updates are only pushed to MAST when there are builds of the operational pipeline, which usually include changes to other subsystems, about every 3 months.

In some cases, users might see patch releases in-between builds of the JWST Operational Pipeline. While these patches may not include changes to the JWST Science Calibration Pipeline, they still impact the calibration products.

The JWST Science Calibration Pipeline can also be updated and released to users before it is integrated into the operational pipeline. These are done via minor releases of the jwst package via conda pip. Such updates will subsequently be included in operations with the next operational build.

This article provides links to release notes for different builds of the operational pipeline and patches affecting the calibration products. When the information is maintained elsewhere, links are provided to that documentation.

Tracking versions in the data products: Several FITS header keywords document the software and reference data versions used to generate data product. These are 2 keywords of note:

  • CAL_VER records the JWST Science Calibration Pipeline version used to process data (e.g., "1.4.6").
  • CRDS_CTX records the context of calibration reference data, in the JWST OPS server, that was used to process data (e.g., "jwst_0857.pmap").



The current build

The current build of the operational pipeline includes build 9.0 —please refer to the latest release notes for additional details.

Known issues

The following issues were not resolved in the jwst calibration pipeline package (version 1.8.2) that was released with build 9.0:

  • Updated in-flight reference data for the NIRCam WFSS mode have not yet been delivered to CRDS, so the pipeline is still using pre-flight calibrations. This usually leads to incorrect computations of spectral trace locations and wavelength assignments during stage 2 calibration, rendering the archive products unsuitable for analysis.
  • The computation of the WCS frame for resampled and combined NIRSpec MOS and fixed slit data can sometimes be in error, leading to problems in late stage processing.

Some step and mode-specific issues will be fixed in build 9.1:

  • NIRSpec IFU cubes often contain sampling artifacts. This problem has recently been resolved and will be included in build 9.1.
  • NIRISS SOSS exposures can occasionally fail in the extract_1d step. Algorithm updates are in progress that will resolve these issues and may be included in build 9.1.
  • The computation of bounding boxes for WFSS sources can sometimes be incorrect and lead to crashes in the extract_2d step. These have been fixed for build 9.1.



Future enhancements for the JWST Science Calibration Pipeline

  • Images obtained with NIRCam subarrays, where the number of reset rows is less than the full amplifier size, have a cyclical bias pattern that is currently not completely removed by the superbias subtraction step. This can be addressed by using new reference files for these subarrays, which are in the process of being delivered by the NIRCam Instrument Definition Team, and corresponding modifications to the pipeline superbias subtraction step for these NIRCam subarrays, which are expected to be implemented in the coming months.
  • The jump detection step for cosmic ray removal in calwebb_detector1 currently runs slowly for cases with 2 or more cosmic rays per pixel ramp. Improvements to the relevant parts of this step have been identified for implementation in the next pipeline release.
  • At very low levels, the NIRCam and NIRISS detectors generally exhibit a behavior referred to as "1/f noise," resulting from slow time-dependent changes that are not fully removed in the reference pixel subtraction step. (Note that NIRSpec detectors are also capable of this behavior, but it is substantially mitigated by use of the IRS2 readout mode which is available for NIRSpec). This effect manifests as slowly varying stripes at low levels along the slow-read direction (e.g., along rows in NIRCam data). Some algorithms already exist to remove this, and are being discussed for implementation in the coming months.


Previous builds

Previous build information as a historical record of the ongoing changes to the JWST Science Calibration Pipeline.




Latest updates
  •  
    added information about build 9.0

  •  
    added information about build 8.1.2

  •  
    added information about build 8.1.1


  • Added known issues with the pipeline found during commissioning
    Added future enhancements information
Originally published