JWST Wide Field Slitless Spectroscopy Pipeline Caveats
Specific characteristics of the JWST Science Calibration Pipeline for wide field slitless spectroscopy (WFSS) modes on NIRCam and NIRISS are described in this article. For example codes or more in depth details, the user should refer to the JWebbinar material on github and to the full JWST calibration pipeline documentation, respectively.
WFSS pipeline steps
Words in bold italics are also buttons
or parameters in GUI tools. Bold
style represents GUI menus/
panels & data software packages.
The default set of observations in WFSS mode includes direct images before and after the grism exposures (see WFSS observation sequence). These direct images are required to enable proper identification of the objects seen in the grism images. Hence, they should be considered when processing the data with the JWST calibration pipeline. Direct images are processed through the calwebb_detector1, calwebb_image2, and calwebb_image3 stages with the aim of deriving a source catalog, which is required for the extraction of the spectra. The grism data are processed through the calwebb_detector1, calwebb_spec2, and calwebb_spec3 stages to obtain 2-D and 1-D calibrated spectra for every source in the catalog.
Direct imaging - resampling step in calwebb_image3 pipeline stage
Note: This paragraph is only relevant to WFSS data taken prior to July 18, 2022, using a DMS Context prior to jwst_0928.pmap (see value of main header keyword CRDS_CTX). Prior to source catalog creation, the direct images for WFSS (which are taken at 2 or more dither positions per blocking filter) are combined into a "_i2d.fits" file by the calwebb_image3 stage of the pipeline after image alignment, sky matching, outlier detection, and resampling onto an undistorted pixel grid. For the default settings of the calwebb_image3 pipeline, this resampling step involves a pixel-by-pixel weighting procedure, using weights that are proportional to the inverse variance of the read noise (VAR_RNOISE) array stored in each input image
(resample.weight_type = 'ivm'). For NIRISS direct imaging data with the WFSS filters that was run through the JWST pipeline using a DMS context prior to jwst_0928.pmap (which was made active on July 18. 2022), this way of resampling imaging data can lead to flux levels of unsaturated point sources in the resampled image that are too low by up to 30% depending on the size of the measurement aperture and the exact positioning of the star centroid relative to pixel boundaries. For such data, we recommend the following workaround to retain proper flux levels in the combined image for NIRISS WFSS data:
my_asn.json is the association file (ASN) for the direct images taken with a given filter.
JWST's near-infrared HgCdTe detectors have a correlated noise, known as 1/f noise, that is introduced by the detector readout system (Moseley et al. 2010). It results in stripes that can be a limiting factor in how faint of a continuum pipeline processed data can reach when the dispersion direction is parallel to the stripes. If the 1/f noise stripes are perpendicular to the dispersion direction, they create faint fluctuations that can mimic emission lines. Examples on how to correct for the 1/f noise can be found in the CEERS JWebbinar notebook or in the TSO JWebbinar notebook.
A source catalog is required to extract the individual spectra from the grism observations. The calwebb_spec2 stage accepts the source catalog obtained by running the source_catalog step in calwebb_image3 on the direct image. The number of sources extracted during the calwebb_spec2 stage is thus limited by the depth and the covered field of view of the direct imaging. The minimal set of columns that need to be provided in the source catalog to run the extraction includes just label and position (as right ascension and declination in degrees).
Summary of the various steps
The spectral extraction happens in the calwebb_spec2 stage of the JWST Science Calibration Pipeline. Example code and description of the individual steps are provided in the notebook "JWST Spectroscopic Data Calibration: Pipeline Stage 2." For the WFSS mode on NIRISS and NIRCam, the steps in the calwebb_spec2 stage happen in the following order: assign_wcs, background, flat_field, extract_2d, wfss_contam (if enabled), photom, and extract_1d.
- The assign_wcs step associates a World Coordinate System (WCS) object with each science exposure. The WCS object transforms positions in the detector frame to positions in a world coordinate frame, ICRS and wavelength.
- The background step subtracts a background reference image from the target exposure. Before being subtracted, the background reference image is scaled to match the signal level of the WFSS image within background (source-free) regions of the image.
- The flat_field step takes an input science dataset and divides it by a flat field reference image. In particular, the SCI array from the flat field reference file is divided into the SCI array of the science dataset, the flat field DQ array is combined with the science DQ array using a bitwise OR operation, and variance and error arrays in the science dataset are updated to include the flat field uncertainty.
- The extract_2d step extracts 2-D arrays from spectral images. The extractions are performed within all of the SCI, ERR, and DQ arrays of the input image model, as well as any variance arrays that may be present. It also computes an array of wavelengths to attach to the extracted data. The extract_2d step uses the source catalog to create the list of objects and their corresponding bounding box. This list is used to make the 2-D cutouts from the dispersed image.
- The wfss_contam step is applied to grism exposures in an attempt to correct effects due to overlapping spectral traces, which often happens in observations of crowded fields. It is not enabled by default. More details are given below.
- The photom step applies flux (photometric) calibrations to a data product to convert the data from units of countrate (DN/s) to surface brightness (MJy/sr). The calibration information is read from a photometric reference file.
- The extract_1d step extracts a 1-D signal from the 2-D cutouts and writes spectral data to an “x1d” product. The extract_1d step collapses the input data from 2-D to 1-D, providing the equivalent of a Boxcar extraction. An additional background subtraction can be done if (and only if) bkg_coeff is given in the "EXTRACT1D" reference file. In that case, the background is determined independently for each column (or row, if dispersion is vertical), using pixel values from all background regions within each column (or row).
These extraction steps are best suited to non-crowded fields. The contamination modeling can be used to mitigate the effects of overlapping traces in crowded fields. WFSS images are not corrected for distortion, but the 1-D extraction accounts for variations across the field when computing the spectral traces and the wavelength calibration.
NIRCam and NIRISS WFSS receive only minimal processing by calwebb_spec3. Essentially, in stage 3, the 2-D data are reorganized to convert stage 2 exposure-based data products to stage 3 source-based data products (exp_to_source step). Subsequently, 1-D spectra are re-extracted by the extract_1d step and the combine_1d step computes a weighted average of 1-D spectra to ultimately write the combined 1-D spectra as output. Example code and description of the individual steps are provided in the notebook JWST Spectroscopic Data Calibration: Pipeline Stage 3.
Background subtraction is the second step of the calwebb_spec2 stage of the JWST Science Calibration Pipeline. This step relies on a single master background (in the form of a reference file), which will be updated with Cycle 1 observations, and assumes that the WFSS background does not vary in time (during an exposure) and its spectrum is constant. This can be a limiting factor in the case of low signal-to-noise observations. As a mitigation strategy, the user can perform a local background subtraction on the extracted 2-D spectra before extracting the 1-D spectra.
One of the main challenges in the spectral extraction for WFSS modes on JWST is the contamination of any individual spectrum by the overlapping spectra of nearby sources. The JWST calibration pipeline is best suited for sparse fields where traces do not overlap and the extraction of 2-D and 1-D spectra is robust. However, observed fields are often crowded. Both NIRISS and NIRCam have grisms that disperse light in 2 orthogonal directions, which could be used to help mitigate some of the spectral overlaps. This is because a spectrum strongly contaminated in one spectral direction might not be as contaminated in the other spectral direction. As fainter magnitudes are reached, however, spectral contamination occurs in any direction and, moreover, being able to correct all spectra for contamination before combining them would allow for the full depth of the data to be used.
The JWST calibration pipeline includes an option to provide a first-order correction for contamination with the wfss_contam step. Briefly, source fluxes from a direct image of the field are used to simulate grism spectra for each source. Each source spectrum is then corrected for contamination by subtracting the simulated spectra of nearby sources. More details are provided on the calibration pipeline documentation. The WFSS contamination correction is not performed by default, but it can be enabled when calling calwebb_spec2 as in this example:
This contamination model provides a first-order correction in the extraction of overlapping spectra, but it is based on single-band photometry (i.e., it does not account for flux variations within a photometric band) and it depends on the depth of the segmentation map created from the direct image during calwebb_image3 processing. Also, some discrepancies have been observed between the modeling and the simulated spectra. Updates to the wfss_contam step will be guided by experience with data obtained on-orbit. Pirzkal et a. (2017) provides more information about contamination modeling for wide field slitless spectra.
Moseley, S. H., et al. 2010 SPIE Proceedings Vol. 7742
Reducing the read noise of H2RG detector arrays: eliminating correlated noise with efficient use of reference signals
Pirzkal, N., et al. 2017 ApJ, 846, 84
FIGS—Faint Infrared Grism Survey: Description and Data Reduction