NIRISS SOSS Known Issues

Known issues specific to time-series observations (TSOs) obtained with NIRISS single object slitless spectroscopy data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. 

On this page


Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRISS SOSS Calibration Status for an overview of the current astrometric, photometric, and target acquisition accuracy of NIRISS SOSS data products.



Artifacts

Information on NIRISS instrument artifacts are found on the main NIRISS Known Issues page.



Pipeline notes

calwebb_detector1 

Reference pixel correction in SOSS

See also: JWST Time-Series Observations Noise Sources

The reference pixel correction step effectively removes several detector-level artifacts from SOSS data, but does not properly remove 1/f noise in extracted spectra. Analyses performed during commissioning show that there is a significant improvement when removing this component using pixels as close as possible to the spectral trace. In general, this is not straightforward to do as most of the NIRISS/SOSS subarrays contain illuminated pixels, but techniques currently in the literature show how to account for this to remove 1/f noise (see, e.g., Radica et al., 2023). For optimal results, it is recommended that such 1/f treatments be performed on individual groups, prior to ramp fitting but after the non-linearity step. Although this sequence should produce optimal results for the read noise-dominated parts of the spectrum, competitive results may be produced by performing the steps in a different order (e.g., after ramp fitting) for other spectral regions.

Saturation levels in NIRISS detectors as applied to SOSS observations

See also: JWST Time-Series Observations Noise Sources

During commissioning TSO observations (proposal ID 1541) of the exoplanet HAT-P-14b, for which saturation was reached by design in the up-the-ramp samples, significant deviations from linear ramps were found even after non-linearity corrections at count levels of about 35,000 ADUs (i.e., about 56,000 e after removal of the superbias). The light curves obtained from pixels exceeding this fluence level exhibited degraded precision. Until the origin of this degradation is better characterized, users are advised to limit the counts in an integration to a maximum of 35,000 ADU/56,000 e.

Quantum yield and pipeline-reported error bars

See also: JWST Time-Series Observations Noise Sources

The quantum yield is an effect over which a given input photon generates more than a single electron (see McCullough et al. 2008). The JWST pipeline does not account for these quantum yields in the noise estimates provided in the number of counts and/or count rates for "rateint.fits" products—it simply assumes the quantum yield is 1. Shortward of 2 μm, the HgCdTe detector in NIRISS might produce more than one photoelectron per photon (see McCullough et al. 2008, Rauscher et al. 2014), resulting in non-Poissonian statistics. For TSOs, this implies that at about 1 μm, larger than expected light curve scatter by a factor of up to 10%–30% could be observed due to this effect. The same enhancement factor at 0.6 μm could be as large as 30%–50%. The magnitude and wavelength dependence of this enhancement are currently under investigation.

calwebb_spec2

SOSS sky background

The sky background associated with SOSS observations has an unusual shape that results from the spectral content of the background (which is typically the zodiacal light), its illumination of the pick-off mirror (POM), and its dispersion by the GR700XD element. The main characteristics include a smooth, raising background level towards longer wavelengths, with a sharp decrease caused by the edges of the POM at about 2.1 μm (which corresponds to column 700 in the trace of order 1). Although the amplitude of the background depends on the strength of the zodiacal signal, its shape as determined during commissioning remains constant to within 2%–3%. The typical peak amplitude of the background flux on the brightest background pixels is approximately 2 ADU/s (i.e., about 3 e/s).

Figure 1. SOSS sky background template (top), along with an example frame obtained during commissioning (middle) and the background-removed frame making use of the SOSS sky background template


The sky background from SOSS can be determined by scaling a model background (top) using background pixels from the target image (middle). Corrections performed by simple scaling remove 97–98% of the background.
This background shape is important to remove in science cases where precise absolute and/or relative flux measurements are performed, such as in exoplanet transit spectroscopy, since the signal from the background can produce significant dilution of an exoplanet’s transit/eclipse depth as a function of wavelength. In general, transit/eclipse depths can be diluted by a factor of about 1 / (1 + FR), where FR is the flux ratio of the background flux over the target flux. Rates of 20 ADU/s from a star at the position of the brightest background pixels implies a dilution of about 90% in the measured transit/eclipse depth.

The JWST calibration pipeline does not currently have a step to remove the background flux, which must be removed manually during a post-processing stage. The NIRISS team has provided a smoothed background measurement obtained during commissioning observations for SUBSTRIP256 and for SUBSTRIP96, obtained by combining and smoothing dithered rates of a field with relatively few stars (observation 5 of program ID 1541).  During commissioning, it was found that scaling this model background frame using background pixels from a target frame, allows the background component to be removed with an accuracy of up to 2%–3% (e.g., peak background rates of 2 ADU/s are reduced to 0.04–0.06 ADU/s). This accuracy, however, might vary by a factor of a few from visit to visit given both, the possibility of zodiacal background variations and the fact that the pupil wheel position doesn't return to the same commanded position for every given visit (Martel, 2022); these effects are currently under investigation. For applications that require higher precision, dithered exposures to obtain background measurements on a nearby area of the sky are encouraged.

SOSS spectral extraction

NIRISS SOSS spectral extraction needs special care at the red end of the spectrum, where order 1 and order 2 partially overlap, in particular for columns below 400 (i.e., above about 2.4 μm as measured by order 1). An algorithm developed by the NIRISS IDT team is implemented in the JWST pipeline to account for this contamination during spectral extraction (the Algorithm to Treat Order ContaminAtion — ATOCA; Darveau-Bernier et al. 2022).

An estimate of this contamination can be obtained using an exposure with the GR700XD grism in combination with the F277W filter. The F277W filter removes contributions from order 2 in the overlap region with order 1. This exposure can be used to obtain a precise shape for the cross-dispersion profile of order 1 in the contamination region, which can subsequently be scaled and subtracted from the CLEAR+GR700XD exposure to provide an uncontaminated estimate of the spectrum in order 2, as well as an estimate of the uncontaminated flux of order 1 in the overlap region.

SOSS wavelength solution

The wavelength solution for NIRISS SOSS observations can shift by up to several pixels between visits due to small GR700XD grism rotations that impart a rotation to the SOSS spectral trace (Known Issues NR-SOSS02 and NR-SOSS03, see table below). Using the PASTASOSS package, both the spectral trace (Baines et al. 2023a) and wavelength solution (Baines et al. 2023b) can be predicted, improving the wavelength solution accuracy to 0.5 pixels. Implementation of this functionality into the JWST calibration pipeline is in development.

calwebb_tso3

Data products from the TSO3 stage of the pipeline have not been fully verified. There are known differences between the extracted 1-D spectrum (*x1d.fits) from the calwebb_spec2 stage of the pipeline and the calwebb_tso3 stage of the pipeline. The step extract_1d is run in both the calwebb_spec2 and calwebb_tso3 stages of the pipeline which produces "*x1d.fits" files from each stage. In calwebb_spec2, the photometric correction (photom step) is applied after the extract_1d step due to the ATOCA algorithm used to disentangle the overlap in orders 1 and 2. In calwebb_tso3, the photom step will have been run before extract_1d which is the default for other observing modes but inappropriate for SOSS data.

Observers are encouraged to use data products from calwebb_spec2 for data analysis.



Summary of common issues and workarounds

The sections above provide detail on each of the known issues affecting NIRISS SOSS data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.

SymptomsCauseWorkaroundFix buildMitigation Plan

NR-SOSS02: The shape of the spectral trace varies from visit to visit for all the NIRISS/SOSS orders. Distortion is higher at longer wavelengths. This might lead to lower SNR in the extracted spectra.

The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the shape of the spectral trace.

Use the PASTASOSS package to predict spectral traces given the PWCPOS, or use the package as a starting point to perform spectral tracing and thus extraction.

N/A

Updated issue

Implementation in the Science Calibration Pipeline is under development.

NR-SOSS03: The wavelength solution varies from visit to visit, appearing to be shifted by a handful of pixels. 

The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the wavelength solution.

Use the PASTASOSS package to predict the wavelength solution for the visit.

 N/A

Updated issue

Implementation in the Science Calibration Pipeline is under development.

NR-SOSS05: Observations show a background that abruptly increases in flux at around column ~700.

This is due to the background (typically dominated by zodiacal light background at NIRISS/SOSS wavelengths) being dispersed by the GR700XD grism into the detector (see Albert et al., 2023; Section 8.5).

Scale the background taken during commissioning for SUBSTRIP256 and for SUBSTRIP96 to observations using non-illuminated regions of the detector (see NIRISS Time Series Observations Pipeline Caveats on the efficiency of this procedure).

N/A

Created issue

Calibration program 4479 is studying background templates at different positions in the sky, as well as the variation of the background due to the pupil wheel position (PWCPOS) varying from visit to visit. Implementation on the JWST pipeline is still under development.

NR-SOSS06: There is evident banding in the cross-dispersion direction, which varies from group-to-group and integration-to-integration.

This is due to residual, uncorrected 1/f noise left over from the reference pixel correction step.

There are several community tools available designed to remove 1/f noise. 

N/A

Created issue

A mitigation plan is under development.

NR-SOSS07: Large number of outliers identified in the jump detection step, which cause lower SNR in the "_rateints" (count rate per integration) products.

The default parameters of the jump step in the JWST pipeline are too aggressive and flag too many pixels as cosmic rays.

Re-run the jump detection step in calwebb_detector1 with a rejection_threshold value > 10 (default is 4.0).

N/A

Created issue

A mitigation plan is under development.

NR-SOSS08: The pipeline throws an error when trying to extract a SOSS spectrum with the GR700XD/F277W combination.

The pipeline currently does not support this grism/filter combination.

There is currently not a workaround.

N/A

Created issue

Updates to the pipeline code to support the GR700XD/F277W combination are under development.

NR-SOSS01: Too many pixels are flagged as "DO_NOT_USE" which are then set to "NaN" values in the science array. Some of these pixels are fine for SOSS data analysis.

Several detector-level reference files (mask, linearity, saturation, super bias) need to be updated to not cause certain pixels to be flagged as "DO_NOT_USE".

Use the attached Jupyter notebook to turn off setting pixels to "DO_NOT_USE".  (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline).

Updated Operations Pipeline

New reference files were delivered to CRDS which no longer erroneously flag good pixels as "DO_NOT_USE". The new linearity and saturation reference files are available in CRDS contexts "jwst_1152.pmap" and later. The new subarray superbias reference files are available in CRDS contexts "jwst_1170.pmap" and later.

Affected data products will be reprocessed; reprocessing of affected data typically takes 2–4 weeks.

NR-SOSS04: The shape of the spectral trace and wavelength solution sometimes varies from exposure to exposure on a given visit, including between subsequent CLEAR and F277W exposures.

The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position. When the position lands above a certain threshold from the commanded position, the onboard mechanism tries to re-adjust the position between exposures until it meets the threshold.

If trying to match CLEAR and F227W exposures, perform a translation and/or rotation of the F227W exposure to match the CLEAR exposure. Use stellar lines to match the wavelength solution on the F277W exposure.

N/A

Updated Operations Pipeline

The NIRISS team has implemented the addition of a "Short First Exposure" consisting of a single group, single integration at the beginning of every exposure that requires an F277W exposure. This will give the on-board mechanism the opportunity to re-adjust the pupil wheel position if needed before the science exposures. This was implemented in the NIRISS SOSS template in APT 2023.5 (release date: August 24, 2023). 



References

Baines, T., et al. 2023a, JWST-STScI-008448
Characterization of the visit-to-visit Stability of the GR700XD Spectral Traces for NIRISS/SOSS Observations

Baines, T., et al. 2023b, JWST-STScI-008571
Characterization of the visit-to-visit Stability of the GR700XD Wavelength Calibration for NIRISS/SOSS Observations

Darveau-Bernier, A., et al. 2022, PASP, 134, 094502
ATOCA: an Algorithm to Treat Order Contamination. Application to the NIRISS SOSS Mode
ADS

Martel, A., 2022, JWST-STScI-008298
The Early Behavior of the NIRISS Pupil Wheel and Filter Wheel

McCullough, P. R., et al. 2008, PASP, 120, 759
Quantum Efficiency and Quantum Yield of an HgCdTe Infrared Sensor Array
ADS

Radica, M., et al., 2023, MNRAS, vol. 524, issue 1
Awesome SOSS: transmission spectroscopy of WASP-96b with NIRISS/SOSS
ADS

Rauscher, B. J., et al., 2014, PASP, vol. 126, issue 942
New and Better Detectors for the JWST Near-Infrared Spectrograph
ADS



Links

Predicting Accurate Spectra Traces in Astrophysical SOSS Spectra (PASTASOSS): https://github.com/spacetelescope/pastasoss




Notable updates


  •  
    Added a section about calwebb_tso3.


Originally published