JWST Operations Pipeline Build 8.1.1 Release Notes

Release notes and known issues in build 8.1 and patch build 8.1.1 of the JWST Operations Pipeline, released on August 19, 2022, are provided in this article.

On this page

This article provides a list of all relevant changes to the operations pipeline that was released with build 8.1 and patch build 8.1.1. This build focused mainly on updates to the JWST Science Calibration Pipeline, but included information relevant to changes in other subsystems that might affect what users see in their data.


Highlights of this build

  • Fixed the units of photometric error estimates in source catalogs created by the stage 3 imaging pipeline.
  • Short-term workaround to handle NIRSpec MOS observations that include source IDs that have negative values or values greater than 10,000.
  • Bug fix that was causing incorrect results in the application of master background subtraction for MIRI MRS observations.


Known issues

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

The following issues were not resolved in the jwst calibration pipeline package (version 1.6.2) that was released with build 8.1.1:

  • The calwebb_image3 and calwebb_spec3 pipelines can occasionally fail with memory allocation errors when processing observations containing very large image mosaics or many hundreds of spectra. Improvements in memory management are underway. Minor improvements are included in build 8.1 and a major update to memory management—especially in the outlier_detection step—will be included in build 9.0.
  • The drizzled images created by the resample and resample_spec steps can contain spatial offsets on the order of 0.5 pixels, due to a bug in the computation of the output WCS grid when using the kernel = square option. This bug has been fixed in the most recent version of the drizzle package and will be included in the build 9.0 release of the pipeline.

Some step and mode-specific issues will be fixed in build 9.0:

  • Incorrect results in the fringe parameters computed for NIRISS AMI observations in the ami_analyze step. 
  • The source_catalog step can produce incorrect source detection results for MIRI imaging data, due to the inclusion of pixels that are not part of the actual imaging portion of the detector. 
  • The tweakreg step only updates the internal gwcs object contained in image datamodels, without updating the FITS WCS keyword values that are included in image headers for convenience. 
  • The skymatch step does not actually subtract the computed sky values from the images when the user selects subtract = True. 

Also, the following infrastructure-related issues affect this build:

  • WFSS program data can produce a significant number of files, so ingestion of data into MAST may be delayed by a few days after completion of the program.

Release notes

See also: GitHub project release notes 1.6.2

Patch build 8.1.1 of the JWST Operations Pipeline was released on August 19, 2022 to the Barbara A. Mikulski Archive for Space Telescopes (MAST). This build includes changes to the JWST Science Calibration Pipeline released in version 1.6.0, 1.6.1, and 1.6.2 of the jwst calibration pipeline package via the PyPI repository. This patch release also includes changes to the JWST Science Data Processing subsystem (SDP) version JWSTDP-2022.3.0, which creates raw (uncal) products.

The following fixes or changes were included in patch build 8.1.1. Changes to specific steps in the pipeline are listed below, along with descriptions to more general changes made to specific packages and related subsystems.

extract_1d step

  • Fixed incorrect variance propagation or computation of errors for the background spectra in all modes, except IFU data that was fixed in build 8.0.1
  • Updated to set the "DO_NOT_USE" DQ flag in wavelength bins of extracted IFU spectra when the extraction aperture has no valid data (usually in the gaps between wavelength channels/bands).

ramp_fitting step

  • Fixed errors in the use of the multiprocessing capability. The error was due to an incorrect handling of an object by astropy.io. This was fixed by handling it entirely in the ramp fitting step code.

resample step

  • Updated code to allow for the parameter values pixfrac, weight_type, kernel, and fillval in the "drizpars" reference file to be loaded when default values in the step are set to None.

residual_fringe step

  • Bug fix to allow the step to run properly on MIRI MRS "long" detector data.

saturation step

  • Fixed the handling of pixels flagged as "NO_SAT_CHECK" in the saturation reference file, so that they are never flagged as saturated.

skymatch step

  • Bug fix to have the computed sky values saved in the images when the input is in the form of an association table.

source_catalog step

  • Bug fix to put error estimates in the same units (Jy) as the photometry for each source.
  • Updated the default parameter values for background box size and minimum number of pixels, in order to avoid "false positives" in the detection of sources.

tweakreg step

  • Updated the step to be more robust against errors that can occur when the source matching parameter values are not a good match to the data.

Pipeline modules

  • The Spec3Pipeline was updated with a short-term fix for NIRSpec MOS observations that have "source_id" values that are negative or >99999, which causes problems for file name construction and parsing. The problematic source IDs are converted to previously unused positive 5-digit values.
  • The Spec3Pipeline and Image3Pipeline were improved to minimize the memory usage during the outlier_detection and resample steps. This change helps with the on-disk processing of hundreds of exposure datasets and complete processing while using less than 30Gb active memory.

General changes

  • Updated the list of allowed dither pattern names in the datamodels schemas to include the new MIRI MRS pattern "SCAN-CALIBRATION", which is used for astrometric calibration observations.

Documentation updates

  • Updated the ramp_fitting step documentation to provide more details on the calculation of variance due to read noise.

JWST Science Data Processing subsystem changes

The JSDP subsystem processes data from the telescope and creates raw (uncal) products. The following changes have been included in this build:

  • The MSA meta data file (jw*_msa.fits) used for processing NIRSpec MOS exposures had an incorrect data type specified for the "source_id" column in the "SHUTTER_INFO" table extension. It had been specified as short integer and has now been updated to be a 4-byte integer, which is necessary to accommodate the expected range of values. Users are advised to download reprocessed MSA meta data files from the archive.
  • Guider "ID" mode products should now have FITS keyword values that reflect the characteristics of the actual guide star used in each ID attempt. Previous products always showed information for the last guide star used.
  • The bug requiring manual intervention to correct the integration time tables for TSO observations should now be resolved.



Notable updates


Originally published