Release notes and known issues in patch build 8.1.2 of the JWST Operations Pipeline, released on September 21, 2022, are provided in this article.

This article provides a list of all relevant changes to the operations pipeline that was released with patch build 8.1.2. This build focused mainly on updates to the JWST Science Calibration Pipeline, but includes information relevant to changes in other subsystems that might affect what users see in their data.

Highlights of this build

• The IFU cube building (cube_build) step has been fixed to properly align and combine IFU observations of moving targets.
• The outlier_detection step has been enhanced to allow the option of saving data to disk during processing, rather than keeping it all in memory, in order to reduce the overall memory usage of the step.
• The tweakreg step has been enhanced to allow for more user control over the details of processing.

Known issues

Release notes

See also: GitHub project release notes 1.7.2

Patch build 8.1.2 of the JWST Operations Pipeline was released on September 21, 2022 to the Barbara A. Mikulski Archive for Space Telescopes (MAST). This build includes changes to the JWST Science Calibration Pipeline released in version 1.7.2 of the jwst calibration pipeline package via the PyPI repository.

The following fixes or changes were included in patch build 8.1.2. Changes to specific steps in the pipeline are listed below, along with descriptions to more general changes made to specific packages and related subsystems. This patch release also includes changes to the JWST Science Data Processing subsystem (SDP) version JWSTDP-2022.3.1, which creates raw (uncal) products.

ami_analyze step

• Removed an implementation that was used to compute the Fourier transform calling the Physical Optics Propagation in PYthon (POPPY) software and reverted to original standalone module in order to avoid further convention change fragility.

assign_wcs step

• Version 0.18.2 of the gwcs package causes an error to be raised when functions used in that package are used to compute FITS Simple Imaging Polynomial (SIP) WCS solutions for unrectified images if the solution does not converge. Updated the assign_wcs step to trap this error and issue a warning, without stopping the pipeline. When this occurs, the FITS SIP solution will not appear in the product headers.
• Updated the step to handle the latest in-flight versions of NIRISS wide field slitless spectroscopy (WFSS) world coordinate system (WCS) transforms.
• Updated the step to populate the "wavelength" extension for MIRI LRS slitless exposures, derived from the WCS solution.
• Modified the default parameters for FITS SIP fitting in the assign_wcs step in order to make this fitting more robust with respect to MIRI imaging distortions.
• This version adds a convenience function update_fits_wcsinfo() to the assign_wcs.util module to allow easy updating of the FITS WCS stored in datamodel.meta.wcsinfo from a datamodel's GWCS. 

cube_build step

• Implemented a fix to the way WCS transforms are used when processing moving target exposures. This ensures that the combined cube is created in the frame of the target, rather than the sky frame.
• Rewrote the code used to set the data quality (DQ) flags for NIRSpec IFU data after discovering the flagging was not done correctly.
• A fix was implemented to address crashes of the code when users selected the output spatial scale for the NIRSpec IFU cube using the --scale1 or --scale2 options.
• Enhancements were implemented to make the step more robust to situations in which there's no good data available for a given wavelength channel.

extract_1d step

• When processing a TSO observation, the extract_1d step normally copies over the timestamp data from the INT_TIMES table to the output "x1dints" product, for use later in constructing light curves. The new ATOCA extraction algorithm used for SOSS data, however, bypassed the function that copies the INT_TIMES data. The code was updated to ensure that the INT_TIMES table extension gets propagated to the output products for NIRISS SOSS mode exposures.

jump step

• Added an enhancement to the step to allow for flagging of groups following a cosmic ray hit. There is evidence to suggest that the slope of the ramp after a hit can be different than the slope preceding the hit.

master_background step

• Implemented an update to fix the handling of the SRCTYPE (source type) value for NIRSpec IFU exposures. In the case of background spectra, this value should be read from the "spec" attribute of the data model and not from the generic attribute.
• Fixed an error in the way the master background spectrum was being applied to MIRI MRS exposures. In this case the sigma-clipped background was not being used as intended for the master background step when the source type is undefined.

outlier_detection step

• Enhanced the step to allow the choice of keeping all images in memory during processing or saving them to disk and only loading when they're needed. This reduces the overall memory requirements. New parameters were added to the step to control this functionality.

photom step

• Fixed a bug that was accidentally applying flux conversion factors for NIRSpec fixed slit exposures twice.

ramp_fit step

• Fixed a bug that allowed output pixels to show up with values of "NaN" and not flagged as bad. The modification now sets output pixels to zero if there's no good data from which to compute a slope and flags them as "DO_NOT_USE" in the DQ array.

resample step

• Fixed an error in the way that variance arrays were being resampled, which had been resulting in many zero-valued pixels in the resampled variance images. This version tracks more carefully the individual pixel contribution and removes from 1/variance computations, pixels with zero weights or zero variance.

skymatch step

• Fixed a bug that was preventing the computed sky from actually being subtracted from the data when requested to do so.

straylight step

• Updated the MIRI MRS stray light correction step to reintroduce the zero point subtraction to remove residual pedestal offsets, of order 0.04 to 0.2 DN/s, that are frequently observed in MRS data. The exact amount of this pedestal varies from program to program, potentially as a consequence of mismatches between the reference darks and the as-observed dark current during a given exposure.
  

tweakreg step

• Updated the functions that construct a source catalog for each input image to ignore pixels flagged as "DO_NOT_USE" or "NON_SCIENCE".
• The parameters used to construct a source catalog for each input image—sharplo, sharphi, roundlo, and roundhi—have been exposed to allow users to adjust the values when desired.
• The parameters used to perform absolute alignment to a reference catalog have been exposed to the user, including abs_minobj, abs_searchrad, abs_use2dhist, abs_separation, abs_tolerance, abs_fitgeometry, abs_nclip, abs_sigma.
• Enhanced the step to allow users to supply their own reference catalog to be used for the absolute alignment phase, in addition to the Gaia catalogs.
• Updated the step to allow a single image to be used as input, in which case only the absolute alignment phase (to a reference catalog) will be performed.
• The step now updates the FITS WCS information (in addition to the WCS datamodel object) at the end of the step, using the computed alignment information. This increases SIP approximation accuracy by a factor of 2.
• Instead of raising an exception, the step now logs an error and proceeds without updating the FITS SIP WCS whenever there is poor convergence in the conversion of GWCS to FITS WCS.
  

wfss_contam step

• Recent updates in the way WCS transforms are stored in the WCS object for WFSS grism images resulted in changes to two attribute names in which those offsets are stored from offset_1 and offset_2 to offset_6 and offset_7. The wfss_contam step has been upgraded to use the new attribute names.

Pipeline modules

• The "pars-spec2pipeline" reference file in CRDS that gets applied to the processing of WFSS exposures has been updated to limit the number of extracted sources to only the 100 brightest objects listed in the input source catalog.

Product associations

• The association generator rules have been updated to work more efficiently and effectively when associating direct images with corresponding grism exposures for WFSS observations. The code now does not consider images that do not share either channel (NIRCam) or pupil (NIRISS) in the constraints.

General changes

• Updated the list of allowed dither pattern names in the datamodels schemas to include the new MIRI pattern "BACKGROUND", which is used for coronagraphic observations.

Documentation updates

• Updated the outlier_detection step documentation to explain the use of the recently implemented parameters that control whether images are held in memory or saved to disk while processing.
• Updated the error propagation documentation to include information on how variances are used during resampling (drizzling).

JWST Science Data Processing subsystem changes

The JSDP subsystem processes data from the telescope and creates raw (uncal) products. The following changes have been included in this build:

• The code was adjusted to use detector-based formulae to calculate values for the transformation from detector to science coordinates, when the aperture defined for the pointing was not found in the SIAF tables. This only affected the NIRSpec WATA target acquisition exposures.
• This build fixes a bug that prevented handling of 2 active guiders in the processing of a vist. Under the bug, if there were 2 active guiders in use for a visit, only the guider functions from Guider1 would be put into the visit timeline and used to generate guider data products. The fix should restore the ability to generate data products for 2 active guiders within the same visit.