NIRISS WFSS Known Issues

Known issues specific to NIRISS wide field slitless spectroscopy data processing in the JWST Science Calibration Pipeline are described below. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. 

On this page

Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRISS WFSS Calibration Status for an overview of the current astrometric, photometric, and target acquisition accuracy of NIRISS WFSS data products.



Artifacts

Information on NIRISS instrument artifacts are found on the main NIRISS Known Issues page.



Pipeline notes

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

The default set of observations in WFSS mode includes direct images before and after the grism exposures (see WFSS observation sequence). These direct images are essential for two key reasons: (1) they are used to determine the absolute astrometry in the grism images, which allows users to register the grism images to a predefined catalog or reference frame (such as existing deep JWST imaging or source catalog) and (2) the wavelength zeropoint and spectroscopic apertures are established in the undispersed frame. Hence, they should be considered when processing the data with the JWST calibration pipeline. Direct images are processed through the calwebb_detector1, calwebb_image2, and calwebb_image3 stages with the aim of deriving a source catalog, which is required for the extraction of the spectra. The grism data are processed through the calwebb_detector1, calwebb_spec2, and calwebb_spec3 stages to obtain 2-D (_cal.fits) and 1-D (_x1d.fits) calibrated spectra for every source in the catalog.

calwebb_spec2

Summary of the various steps

The spectral extraction happens in the calwebb_spec2 stage of the JWST Science Calibration Pipeline. Descriptions of the individual steps and example code for running these steps are provided in the notebook "JWST Spectroscopic Data Calibration: Pipeline Stage 2." 

The steps in the calwebb_spec2 stage happen in the following order: assign_wcs, background, flat_field, extract_2d, wfss_contam (if enabled), photom, and extract_1d

  • The assign_wcs step associates a world coordinate system (WCS) object with each science exposure. The WCS object transforms positions in the detector frame to positions in the world coordinate frame, ICRS and wavelength.
    • Note: for pure parallel observations, the WCS is incorrect (see Known Issue GI06 in the table below). While a pipeline implementation to fix this error is in development, a script is available to correct the WCS information in the headers of pure parallel observations.
  • The background step subtracts a background reference image from the target exposure. Before being subtracted, the background reference image is scaled to match the signal level of the WFSS image within background (source-free) regions of the image.

  • The flat_field step takes an input science dataset and divides it by a flat field reference image. In particular, the SCI array from the flat field reference file is divided into the SCI array of the science dataset, the flat field DQ array is combined with the science DQ array using a bitwise OR operation, and variance and error arrays in the science dataset are updated to include the flat field uncertainty.

  • The extract_2d step excises 2-D arrays from the spectral images for each source. The excisions are performed within all of the SCI, ERR, and DQ arrays of the input image model, as well as any variance arrays that may be present. It also computes an array of wavelengths to attach to the extracted data. The extract_2d step uses the source catalog to create the list of objects and their corresponding bounding box. This list is used to make the 2-D cutouts from the dispersed image. 

  • The wfss_contam step is applied to grism exposures in an attempt to correct effects due to overlapping spectral traces, which often happens in observations of crowded fields. It is not enabled by default. More details are given below. 
    • Note: while there is an algorithm in the pipeline to model contamination from overlapping spectral traces, it has not yet been validated for NIRISS WFSS spectra. Investigation to validate this step is on-going.

  • The photom step applies flux (photometric) calibrations to a data product to convert the data from units of count rate (DN/s) to surface brightness (MJy/sr). The calibration information is read from a photometric reference file.

  • The extract_1d step extracts a 1-D signal from the 2-D cutouts and writes spectral data to an “x1d” product. The extract_1d step collapses the input data from 2-D to 1-D, providing the equivalent of a boxcar extraction. An additional background subtraction can be done if (and only if) bkg_coeff is given in the "EXTRACT1D" reference file. In that case, the background is determined independently using pixels in the cross-dispersion axis.

These extraction steps are best suited to non-crowded fields. The contamination modeling, when enabled, can be used to mitigate the effects of overlapping traces in crowded fields. WFSS images are not corrected for distortion, but the 1-D extraction accounts for variations across the field when computing the spectral traces and the wavelength calibration.

Background subtraction

Background subtraction is the second step of the calwebb_spec2 stage of the JWST Science Calibration Pipeline. This step relies on a single master background (in the form of reference file "wfssbkg") and assumes that the background is constant in amplitude and spectral shape. This can be a limiting factor in the case of low signal-to-noise observations. As a mitigation strategy, the user can perform a local background subtraction on the extracted 2-D spectra before extracting the 1-D spectra.

Note that investigation to improve the background reference file, including updated modeling of the NIRISS light saber, is on-going.

Contamination modeling

One of the main challenges in the spectral extraction for WFSS modes on JWST is the contamination of any individual spectrum by the overlapping spectra of nearby sources. The NIRISS GR150 grisms disperse light in 2 orthogonal directions, which could be used to help mitigate some of the spectral overlaps. This is because a spectrum strongly contaminated in one spectral direction might not be as contaminated in the other spectral direction. As fainter magnitudes are reached and more objects are detected, spectral contamination is more likely to occur in both directions and, moreover, being able to correct all spectra for contamination before combining them would allow for the full depth of the data to be used.

The JWST Science Calibration Pipeline includes an option to provide a zeroth-order correction for contamination with the wfss_contam step, though this step is not yet scientifically validated. Briefly, source fluxes from a direct image of the field are used to simulate grism spectra for each source. Each source spectrum is then corrected for contamination by subtracting the simulated spectra of nearby sources. More details are provided on the calibration pipeline documentation. The WFSS contamination correction is not performed by default since it is not yet validated, but it can be enabled when calling calwebb_spec2 as in this example:

from jwst.pipeline import Spec2Pipeline
spec2 = Spec2Pipeline()
spec2.wfss_contam.skip = False

This contamination model provides a first-order correction in the extraction of overlapping spectra, but it is based on single-band photometry (i.e., it does not account for flux variations within a photometric band) and it depends on the depth of the segmentation map created from the direct image during calwebb_image3 processing. Also, some discrepancies have been observed between the modeling and the simulated spectra. Updates to the wfss_contam step will be guided by experience with data obtained on-orbit.

The scientific community have largely identified three ways of mitigating contamination by additional modeling, which require incorporating additional data to break the fundamental degeneracy. Pirzkal et a. (2017) employ broadband photometry to move beyond a zeroth order model to a more complex model. Ryan, Casertano & Pirzkal (2018) use WFSS data at additional orients to solve the linear-system of equations that describe the flux in the WFSS images. Brammer (2023) use SED models and non-linear modeling to determine the contamination-free model spectra and parameters (e.g., redshift, line flux, etc.).

calwebb_image3

Source catalog

A source catalog is required to extract the individual spectra from the grism observations. The calwebb_spec2 stage accepts the source catalog obtained by running the source_catalog step in calwebb_image3 on the direct image. The number of sources extracted during the calwebb_spec2 stage is thus limited by the depth and the covered field of view of the direct imaging. The minimal set of columns that need to be provided in the source catalog to run the extraction includes just label and position (as right ascension and declination in degrees). The quality of the extracted 1-D spectra depend on a reliable source catalog with precise coordinates and morphologies (i.e., point versus extended).



calwebb_spec3

WFSS receives only minimal processing by calwebb_spec3. In this stage, the 2-D data are reorganized to convert stage 2 exposure-based data products to stage 3 source-based data products (exp_to_source step). Subsequently, 1-D spectra are re-extracted by the extract_1d step and the combine_1d step computes a weighted average of 1-D spectra to ultimately write the combined 1-D spectra as output. Descriptions of the individual steps and example code for running each step are provided in the notebook JWST Spectroscopic Data Calibration: Pipeline Stage 3.



Summary of common issues and workarounds

The sections above provide detail on each of the known issues affecting NIRISS WFSS data; the table below summarizes some of the most likely issues users may encounter along with any workarounds, if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.

SymptomsCauseWorkaroundFix buildMitigation Plan

NR-WFSS01: Spectral traces are offset from the sources in the cross-dispersion direction.

The current "specwcs" reference file (as of CRDS version jwst_1176.pmap) incorporates the field dependence in the wavelength dispersion solution. However, the shape of the spectral traces also vary as a function of detector position. 

As of CRDS version jwst_1228.pmap, the field dependence in the spectral trace shape is calibrated for the GR150R/F200W and GR150C/F200W combinations.

None.


Updated issue

In-depth investigation is underway to calibrate the change in shape of the spectral traces as a function of detector position for the remaining filter/grism pairs.

NR-WFSS02: For direct imaging: when the peak pixel of a star (or other compact object) reaches beyond ~25,000 ADU in an integration (after bias and dark subtraction and linearity correction), it starts "spilling" charge to its neighboring pixels. This causes an effective "widening" of the PSF or charge distribution, as well as flux loss for those objects in the combined, resampled products of the calwebb_image3 stage of the Science Calibration Pipeline. This effect is strongest for the most undersampled modes (i.e., filters with pivot wavelength <= 2 μm).

This is due to the so-called "brighter-fatter effect" (BFE) that affects near-IR H2RG detectors, in combination with the current way "jumps" are detected and dealt with in the calwebb_detector1 pipeline stage, and how the latter affects image combination in the calwebb_image3 pipeline stage.

No efficient workaround is available at present.


Updated Operations Pipeline

Apply a new Science Calibration Pipeline step called charge_migration within the calwebb_detector1 stage (Goudfrooij et al. 2024). STScI is reprocessing affected data products with an updated Operations pipeline, installed on December 5, 2023.  Reprocessing of affected data typically takes 2–4 weeks after the update.

NR-WFSS03: There is an offset in the wavelength zeropoint that is most pronounced in the F200W filter.

The filteroffset reference files, which accounts for filter-to-filter offsets in the astrometric solution, does not include WFSS exposures.



Updated Operations Pipeline 

Updated filter offset files that include WFSS exposures were delivered to CRDS and are available as of CRDS version jwst_1176.pmap and later.

NR-WFSS04: WCS keywords in the header of pure parallel WFSS observations (both direct images and grism exposures) taken prior to June 12, 2024, are "off" by of order 0.1 arcsec no matter which SI is prime, and the offset is generally different for different dither positions. 

In JWST data, the header keywords that are used to determine precise WCS information use the guide star coordinates. In pure parallel data taken prior to June 12, 2024, these coordinates were unknown prior to execution. This is why for those pure parallel data, the WCS were determined by using a "coarse" algorithm, leading to the accuracy issue.

For the data affected by this issue, a Python script has been made available in this GitHub repository to correct the relevant WCS header keywords in the products of the “calwebb_detector1” pipeline, i.e., the “_rate.fits” and/or “_rateints.fits” files. After that script has been run on the input file(s), one can then (re-)run Stage 2 of the JWST pipeline (“calwebb_image2” and/or “calwebb_spec2”) on those files, which will then have correct WCS header information.

10.2 (for data taken after June 12, 2024)

Updated Operations Pipeline 

This issue was fixed in JWST operations as part of build 10.2 of the JWST Science Calibration Pipeline (June 12, 2024). Data taken after that date do not have this issue anymore.



References

Brammer, G. 2023, Zenodo
grizli

Goudfrooij, P. et al. 2024, PASP, 136, 014503
An Algorithm to Mitigate Charge Migration Effects in Data from the Near Infrared Imager and Slitless Spectrograph on the James Webb Space Telescope

Moseley, S. H., et al. 2010 SPIE Proceedings Vol.  7742
Reducing the read noise of H2RG detector arrays: eliminating correlated noise with efficient use of reference signals

Pirzkal, N., et al. 2017 ApJ, 846, 84
FIGSFaint Infrared Grism Survey: Description and Data Reduction

Ryan, R. E., Casertano, S., Pirzkal, N. 2018, PASP 130, 034501
Linear: A Novel Algorithm for Reconstructing Slitless Spectroscopy from HST/WFC3




Notable updates
  •  
    Resolved issue NR-WFSS04: WCS keywords in the header of pure parallel WFSS observations (both direct images and grism exposures) taken prior to June 12, 2024, are "off" by of order 0.1 arcsec no matter which SI is prime, and the offset is generally different for different dither positions. 

  •   
    Updated issue NR-WFSS01: the field dependence of the spectral trace shape is now calibrated for both grisms in the F200W filter.

  •  
    Added issue NR-WFSS04: WCS keywords in the header of pure parallel WFSS observations are "off" by about 0.1 arcsec
Originally published