Release notes and known issues in build 9.1 and patch builds 9.1.1 and 9.1.2 of the JWST Operations Pipeline, released on March 15, 2023, are provided in this article.

This article provides a list of all relevant changes to the operations pipeline that was released with build 9.1 and patch builds 9.1.1 and 9.1.2. This build focused mainly on updates to the JWST Science Calibration Pipeline, but includes information relevant to changes in other subsystems that might affect what users see in their data. 

Build 9.1 and the 9.1.1 patch were released together on February 28, 2023, followed shortly after by patch 9.1.2 on March 15, 2023.

Highlights of this build

Build 9.1 and patch 9.1.1

• Words in bold are GUI menus/
  panels or data software packages; 
  bold italics are buttons in GUI
  tools or package parameters.

  Updated the ramp_fitting and some other downstream steps to assign "NaN" to pixels for which a count rate cannot be computed, instead of zero.
• Fixed the handling of NIRSpec "leakcal" exposures, including updates to the way they are associated with both target and background exposures in association files, and the way they are subtracted during calwebb_spec2 pipeline processing.
• Fixed issues with the computation of the output WCS grid for resampled NIRSpec MOS and fixed slit data.
• Fixed bugs in the application of flux calibration to NIRSpec IFU extended source data and the construction of NIRSpec IFU cubes.
• Updated the NIRISS SOSS "Algorithme de Traitement des Orders ContAminée" (ATOCA) algorithms used for creating 1-D extracted spectra.
• Adjusted the level1b FITS keyword values in science data products for TARG_RA and TARG_DEC to account for proper motion of the target.

Patch 9.1.2

• The opencv-python package was added to the pipeline requirements to support the new snowball and shower correction^* in the jump step.
• Fixed an issue causing linked background exposures to not always get associated with their corresponding science exposures.

Known issues

On April 12, 2023, the deprecated codecov package was removed  from PyPI. Users attempting to install a specific DMS build using a specific tag for a particular build will see the installation fail. Users should use the generic methods provided in GitHub for Installing the latest releases or as instructed in the JWST Pipeline Complete Setup Instructions. This issue will be addressed in build 9.2.

The following issues were not resolved in the jwst calibration pipeline package (version 1.9.6) that was released with build 9.1:

• Updated inflight reference data for the NIRCam WFSS mode are still undergoing testing and hence have not been installed in CRDS, so the pipeline is still using pre-flight calibrations. This usually leads to incorrect computations of spectral trace locations and wavelength assignments during stage 2 calibration, rendering the archive products unsuitable for analysis.

Some step and mode-specific issues will be fixed in build 9.2:

• NIRSpec IFU Outlier Detection (in the stage 3 or calwebb_spec3 pipeline) can have anomalous results, which is now known to be due a problem in the WCS transforms included in one of the IFU WCS reference files in CRDS.
• NIRSpec fixed slit data does not get the proper point source vs. extended source flux calibrations applied to each slit instance in the stage 2 photom step. All slits are being treated as point source data.
• The stage 2 and 3 resample step, which uses drizzle, has a bug that can cause regions of the input images to not appear in the resampled image, under certain circumstances.
• The snowball and shower correction in the jump step has a couple of bugs that have already been resolved in the development version of the pipeline, but further testing is underway to ensure the step is working as expected. 

Data affected by this build

Provided below is a list of products in MAST generated by the JWST Science Calibration Pipeline that may be most significantly affected by the updates in build 9.1 and patches 9.1.1 and 9.1.2. The changes listed in the release notes may affect other modes as well, but many of these will be transparent to the MAST user.

The data and pipeline stage affected for data in MAST are:

• NIRSpec IFU observations that include "leakcal" exposures - stage 2
• NIRSpec MOS observations that select "Master Background" option in APT - stage 2
• NIRSpec MOS observations using multiple shutter slitlets together with multiple point nodding dithers - stage 2
• NIRSpec MOS and fixed slits with multiple images - stage 2
• NIRISS SOSS observations - stage 2
• NIRCam and NIRISS WFSS data - stage 2
• MIRI image data with linked background exposures - stage 2 and 3
• All science data using a target with significant proper motion. The "_uncal" data needs to be regenerated using this build and archived - stage 0
• All science data that could be affected by saturated pixels - stage 1

Release notes

See also: GitHub jwst project release notes 1.9.6

Build 9.1 of the JWST Operations Pipeline was released on March 15, 2023 to the Barbara A. Mikulski Archive for Space Telescopes (MAST). This build includes changes to the JWST Science Calibration Pipeline released in version 1.9.6 of the jwst calibration pipeline package via the PyPI repository.

The following fixes or changes were included in build 9.1. Changes to specific steps in the pipeline are listed below, along with descriptions to more general changes made to specific packages and related subsystems. This release also includes changes to the JWST Science Data Processing subsystem (SDP) version JWSTDP-2022.5.0 which creates raw (uncal) products.

assign_wcs step

• Fixed a bug in the computation of bounding box corners for wide field slitless spectroscopy (WFSS) 2-D source regions. The error occurred when the calibration code tried to extract source/spectrum regions that have zero or less than 1 extent in one or more dimensions.
• Changed how the NIRSpec MSA configuration data (from the MSA metadata file) is ingested for observations using multiple shutter slitlets together with multiple point nodding dithers. The sofware now retrieves the source_id for a given slitlet from the primary shutter, rather than just taking it from the first shutter in the slitlet (which can now be zero).
• Added approximated imaging FITS World Coordinate System (WCS) solution keywords to the headers of WFSS grism images when they receive stage 2 (calwebb_spec2) processing, to aid in the use of offline processing and analysis tools that rely on FITS WCS information.

combine_1d step

• Updated this step to sort the combined wavelength array before building the spectral WCS, to avoid occasional issues with non-monotonic wavelengths.

cube_build step

• Fixed a bug in the code that prevented users from specifying desired min/max wavelength values for the output cube for NIRSpec IFU observations.
• Fixed a bug in the 3D drizzle method that was leading to sampling artifacts in NIRSpec IFU cubes, due to incorrect footprint and WCS construction.
• Changed the fill value from 0 to "NaN" for output cube spaxels that have no good data. This prevents confusing these pixels with those for which 0 is a valid value for the count rates.
• Removed the code that trimmed empty wavelength planes from output cubes, so that all cubes will have a consistent size. Empty wavelength planes will contain "NaN" fill values.
• Fixed several memory management bugs in the underlying C code that performs the actual drizzling of data.

extract_1d step

• Updated the IFU spectral extraction algorithms to properly handle the "NaN" fill values that are now contained in input IFU cubes.
• Major upgrades to the NIRISS SOSS ATOCA extraction algorithms provided by the NIRISS team to improve robustness and speed. The upgrade handles several failure modes, stores a wavelength grid in a new datamodel, and increases control via additional step parameters.

extract_2d step

• Fixed a bug in the slice limits used in the extraction of WFSS 2-D source cutouts to avoid problems with spectra having near-zero length.
• Updated the step to add to the metadata of each WFSS source extraction the slit-specifc keywords SRCRA and SRCDEC.

flatfield step

• Updated the computation of the output "ERR" array for FGS guide star data to combine the input "ERR" and flat field "ERR" in quadrature.

guider_cds step

• Updated this step to calculate and output "ERR" array values for FGS guide star data based on the readnoise variance.

imprint step

• Updated to ensure that the proper "leakcal" (a.k.a. imprint) exposures are subtracted from each target and background exposure by matching them according to observation number and dither position index.

jump step

• The opencv-python package was added to the pipeline requirements to support the new snowball and shower correction in the jump step; however, the snowball and shower correction is turned off in the operations pipeline while further testing is completed. 

photom step

• Updated the handling of the pixel area data array so that only a cutout matching the subarray of the science data is stored in the output "AREA" extension.
• Fixed a bug in the application of flux calibration to NIRSpec IFU extended source data that was inadvertently dividing by the pixel area twice. It also sets the output units correctly to "sr" instead "arcsec²."

ramp_fitting step

• Updated the step so that pixels for which a slope value cannot be computed (i.e., no good data) are now set to "NaN" in the output products, rather than zero. Such pixels are also flagged with "DO_NOT_USE".
• Updated the propagation of saturation flags to the output products (_rate and _rateints) so that the flag only appears in those products if all groups for a given integration or exposure are saturated. Partially-saturated pixels will no longer have the saturation flag set in the output products.
• Fixed a bug for ramp segments that contain only 2 good groups and the 2nd group is flagged with "JUMP_DET" (jump detection), so that the resulting slope is set to "NaN" (instead of zero) and flagged as "DO_NOT_USE".

resample_spec step

• Fixed bugs in the construction of the spectral output WCS for NIRSpec slit instances (MOS and fixed slit), which were leading to incorrect WCS objects for resample data or crashes during resampling.

tweakreg step

• Updated the step to no longer skip the step if there is only one input image, so that operations like alignment to a reference catalog can be performed on a single image.
• Fixed a bug in the computation of the 2-D histogram bin size, which is used in pre-alignment source matching.

Pipeline modules

• The calwebb_spec2 pipeline module was updated to subtract NIRSpec "leakcal" exposures from both target and background exposures before applying background subtraction.

Product associations

• The association generator rules for NIRSpec observations with "leakcal" exposures were updated to fix a bug to properly include all "leakcal" exposures taken with both science and background targets.
• Enhanced and made fixes to the rules that associated background exposures with science target exposures.
• Updated the rules for NIRISS WFSS observations that use both grisms (GR150R and GR150C) so that separate stage 3 (calwebb_spec3) association files and products are created for each grism, rather than trying to combine them.
• Fixed a bug in the creation of NIRSpec IFU associations for observations that use point source nodding to accomplish background subtraction.
• Fixed a bug in building the correct "science" and "background" exposure assignments for calwebb_spec2 associations for MIRI LRS fixed slit observations using the "along-slit-nod" dither pattern, which uses the alternate nod position as background for each exposure.
• Ensure all NIRSpec imprint exposures are correctly included in associations.
• Ensure matched exposures generated by list of association candidates continue processing, in order to avoid linked exposures—like backgrounds—from getting dropped from associations.

General changes

• The CRDS_PUB server was deprecated and all references to the server were removed.
  
• Updated the datamodels schema to replace the old subarray "SUB320ALWB" with the new value "SUB400X256ALWB" in the list of allowed values for NIRCam.
• Updated the datamodels schema for slit-based spectra (e.g., MIRI LRS fixed slit) to include the flat field variance data array, so that it shows up with proper values in resampled (_s2d) and extracted (_x1d) products.
• Add opencv-python to pipeline requirements to support the snowball correction in the jump step.

JWST Science Data Processing subsystem changes

The JSDP subsystem processes data from the telescope and creates raw (_uncal) products. The following changes have been included in this build:

• Adjusted the Level1b FITS keyword values in science data products for TARG_RA and TARG_DEC to account for proper motion of the target.

• Several keywords in guide star products were added or fixed. In particular, the values for the keywords DETXCOR, DETYCOR, DETXSIZ, and DETYSIZ are assigned from engineering data. These keywords are used to verify the subarray columns and rows size and to calculate the subarray starting points in in the science frame for all the gs-acq1, gs-acq2, gs-fg, and gs-track FITS file headers.

• Updated the version of Guide Star Catalog used to fill the guide star characteristic in the FITS keywords to GSC2421.