JWST Calibration Pipeline Caveats

JWST Science Calibration Pipeline software is under continual development and improvement. At any time, there may be limitations to the pipeline products for certain instrument modes and science data types. This article provides an overview and links to other articles as more information becomes available.

July 2022: At this early stage of the mission, with the current operational pipeline and available reference files, calibration products may show a variety of known issues that produce poor registration of images, double spectral lines, or other effects that may limit their usefulness for scientific analysis. Also, users should note that early data available from MAST may have been processed with ground data or model-based reference files; science data will require reprocessing over time to produce the best science-quality results. These issues are being addressed as quickly as possible. Instrument teams will be working in the next weeks to finish updating the reference files with in-flight data obtained during commissioning and later on with data obtained during Cycle 1 observations.

The development team will be working on updates to the JWST Science Calibration Pipeline and will provide information on plans for these fixes in this article: JWST Operational Pipeline Build Information. In the mean time, users should refer to the articles below and contact the JWST Help Desk as needed for assistance.


Tracking the pedigree of data products: Several FITS header keywords document the software and reference data versions used to generate data product. These are 2 keywords of note:

Check the JDox General Pipeline Processing Caveats page for known issues in these or subsequent versions. If an issue that impacts your science was addressed in a later version, download the latest version of the data or reprocess the data locally.

In your publications, please be sure to mention the CAL_VER and CRDS_CTX values of the JWST data used in your data analysis. That way, future researchers can reproduce results and retrospectively assess possible calibration issues or reprocessing needs.


On this page

See also: JWST Operational Pipeline Build Information

The JWST Science Calibration Pipeline software performs nominal processing of JWST data for the instruments and instrument modes, making science-ready data products available to users. The pipeline software is expected to be dynamic, with the scheduling of new software builds as experts implement improvements to the current algorithms or address features and artifacts found during in-flight operations. However, even with these improvements, it is impossible to accommodate certain specialized data processing schemes within the context of a standard pipeline processing system, and so, some users may need to adapt various parameters in the software to their particular needs.

The articles mentioned below attempt to convey some of the limitations in standard processing and reference files used, and suggest options that users of various observing modes need to know as they consider the best way to optimize their JWST data processing. They also provide information about the schedule for software updates that are planned to address these limitations, along with known bugs that may be encountered when analyzing data. These articles will be updated regularly as new information becomes available about JWST data.

Check the "Latest Update" box at the bottom of each article to see when the last article updates were made.



JWST Operational Pipeline build information

The JWST Science Calibration Pipeline used by the Mikulski Archive for Space Telescopes (MAST) to calibrate all JWST data is a subsystem of the JWST Operational Pipeline. It is updated on a regular basis. These updates are only pushed to MAST when there are builds of the operational pipeline, which usually include changes to other subsystems, about every 3 months. In some cases, users might see patch releases in-between builds of the JWST Operational pipeline. While these patches may not include changes to the JWST Science Calibration Pipeline, they still impact the calibration products.

Please refer to this article, JWST Operational Pipeline Build Information, for specific information about current and previous calibration pipeline software build deliveries, as well as known issues or shortcomings of each build and planned enhancements.



Non-instrument or mode-specific pipeline caveats 

This article covers pipeline caveats for general data processing that may be applicable to all or multiple instruments and modes: JWST General Calibration Pipeline Caveats



Instrument mode-specific pipeline caveats 

The articles listed below provide information on pipeline caveats for several different instruments and modes:

Additional articles will be available at a later date.



Status of reference files informed by in-flight performance

Each stage of the JWST Science Calibration Pipeline uses a set of instrument-specific reference files that ensure the science calibration pipeline meets its accuracy requirements. Before launch, instrument teams used dummy or ground testing data to support development of the science calibration pipeline stages. During commissioning, teams started updating the reference files using information learned from in-flight performance. This process is ongoing and not all reference files were updated from their pre-flight versions prior to the end of commissioning. Data downloaded from MAST that were calibrated with pre-flight reference files may suffer from poor calibration accuracy.

Users are encouraged to check whether the data downloaded from MAST have been processed with reference files informed by in-flight performance. Calibration reference files are stored and managed within the Calibration Reference Data System (CRDS) and used to reprocess the affected data. Users can check the reference file context for the processing of their data using the CRDS_CTX header keyword. The table below provides the earliest CRDS context version number associated with all in-flight reference files for each observing mode. If the CRDS context in the CRDS_CTX  header keyword is an earlier version, users are encouraged to re-run the science calibration pipeline with the latest CRDS context version, as described in this science calibration pipeline page


Table 1. Minimum CRDS context versions that use reference files informed by in-flight data

MIRI
Observing modeCRDS context in JWST OPS server
Imaging (FASTR1 readout; FULL frame)jwst_0913.pmap
Imaging (SLOWR1 readout; subarrays)reference file updates in progress
Coronagraphic imagingreference file updates in progress
Medium resolution spectroscopy (FASTR1 readout)jwst_0913.pmap
Medium resolution spectroscopy (SLOWR1 readout)reference file updates in progress
Low resolution spectroscopy

jwst_0927.pmap

NIRCam
Observing modeCRDS context
Imagingreference file updates in progress
Coronagraphic imagingreference file updates in progress
Time-series imagingreference file updates in progress
Grism time seriesreference file updates in progress
Wide field slitless spectroscopyreference file updates in progress
NIRISS
Observing modeCRDS context
Wide field slitless spectroscopy

jwst_0950.pmap*

* There is a minor shift in the wavelength solution between these calibration reference files and those measured in flight. The wavelengths in the calibrated outputs are offset by ~0.013 µm for GR150C and ~0.007 µm for GR150R. A new "SPECWCS"  reference file that has this offset corrected is currently being developed.

Single object slitless spectroscopy

jwst_0923.pmap

Aperture masking interferometryjwst_0895.pmap
Imagingjwst_0895.pmap
NIRSpec
Observing modeCRDS context
Multi-object spectroscopyreference file updates in progress
IFU spectroscopyreference file updates in progress
Fixed slit spectroscopyreference file updates in progress
Bright object time-series spectroscopyreference file updates in progress



Latest updates
  •  
    Updated CRDS Context for NIRISS WFSS


  • Updated MIRI LRS reference file context to jwst_0927.pmap  
Originally published