NIRISS SOSS Known Issues
Known issues specific to time-series observations (TSOs) obtained with NIRISS single object slitless spectroscopy data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science.
On this page
Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.
Please also refer to NIRISS SOSS Calibration Status for an overview of the current astrometric, photometric, and target acquisition accuracy of NIRISS SOSS data products.
Artifacts
Information on NIRISS instrument artifacts are found on the main NIRISS Known Issues page.
Pipeline notes
calwebb_detector1
Reference pixel correction in SOSS
See also: JWST Time-Series Observations Noise Sources
The reference pixel correction step effectively removes several detector-level artifacts from SOSS data, but does not properly remove 1/f noise in extracted spectra. Analyses performed during commissioning show that there is a significant improvement when removing this component using pixels as close as possible to the spectral trace. In general, this is not straightforward to do as most of the NIRISS/SOSS subarrays contain illuminated pixels, but techniques currently in the literature show how to account for this to remove 1/f noise (see, e.g., Radica et al., 2023). For optimal results, it is recommended that such 1/f treatments be performed on individual groups, prior to ramp fitting but after the non-linearity step. Although this sequence should produce optimal results for the read noise-dominated parts of the spectrum, competitive results may be produced by performing the steps in a different order (e.g., after ramp fitting) for other spectral regions.
Saturation levels in NIRISS detectors as applied to SOSS observations
See also: JWST Time-Series Observations Noise Sources
During commissioning TSO observations (proposal ID 1541) of the exoplanet HAT-P-14b, for which saturation was reached by design in the up-the-ramp samples, significant deviations from linear ramps were found even after non-linearity corrections at count levels of about 35,000 ADUs (i.e., about 56,000 e– after removal of the superbias). The light curves obtained from pixels exceeding this fluence level exhibited degraded precision. Until the origin of this degradation is better characterized, users are advised to limit the counts in an integration to a maximum of 35,000 ADU/56,000 e–.
Quantum yield and pipeline-reported error bars
See also: JWST Time-Series Observations Noise Sources
The quantum yield is an effect over which a given input photon generates more than a single electron (see McCullough et al. 2008). The JWST pipeline does not account for these quantum yields in the noise estimates provided in the number of counts and/or count rates for "rateint.fits" products—it simply assumes the quantum yield is 1. Shortward of 2 μm, the HgCdTe detector in NIRISS might produce more than one photoelectron per photon (see McCullough et al. 2008, Rauscher et al. 2014), resulting in non-Poissonian statistics. For TSOs, this implies that at about 1 μm, larger than expected light curve scatter by a factor of up to 10%–30% could be observed due to this effect. The same enhancement factor at 0.6 μm could be as large as 30%–50%. The magnitude and wavelength dependence of this enhancement are currently under investigation.
calwebb_spec2
SOSS sky background
The sky background associated with SOSS observations has an unusual shape that results from the spectral content of the background (which is typically the zodiacal light), its illumination of the pick-off mirror (POM), and its dispersion by the GR700XD element. The main characteristics include a smooth, rising background level towards longer wavelengths, with a sharp decrease caused by the edges of the POM at about 2.1 μm (which corresponds to column 700 in the trace of order 1). Although the amplitude of the background depends on the strength of the zodiacal signal, its shape as determined during commissioning remains constant to within 2%–3%. The typical peak amplitude of the background flux on the brightest background pixels is approximately 2 ADU/s (i.e., about 3 e–/s).
The JWST calibration pipeline does not currently have a step to remove the background flux, which must be removed manually during a post-processing stage. The NIRISS team has provided a smoothed background measurement obtained during commissioning observations for SUBSTRIP256 and for SUBSTRIP96, obtained by combining and smoothing dithered rates of a field with relatively few stars (observation 5 of program ID 1541). During commissioning, it was found that scaling this model background frame using background pixels from a target frame, allows the background component to be removed with an accuracy of up to 2%–3% (e.g., peak background rates of 2 ADU/s are reduced to 0.04–0.06 ADU/s). This accuracy, however, might vary by a factor of a few from visit to visit given both, the possibility of zodiacal background variations and the fact that the pupil wheel position doesn't return to the same commanded position for every given visit (Martel 2022); these effects are currently under investigation. For applications that require higher precision, dithered exposures to obtain background measurements on a nearby area of the sky are encouraged.
SOSS spectral extraction
NIRISS SOSS spectral extraction needs special care at the red end of the spectrum, where order 1 and order 2 partially overlap, in particular for columns below 400 (i.e., above about 2.4 μm as measured by order 1). An algorithm developed by the NIRISS IDT team is implemented in the JWST pipeline to account for this contamination during spectral extraction (the Algorithm to Treat Order ContaminAtion — ATOCA; Darveau-Bernier et al. 2022).
An estimate of this contamination can be obtained using an exposure with the GR700XD grism in combination with the F277W filter. The F277W filter removes contributions from order 2 in the overlap region with order 1. This exposure can be used to obtain a precise shape for the cross-dispersion profile of order 1 in the contamination region, which can subsequently be scaled and subtracted from the CLEAR+GR700XD exposure to provide an uncontaminated estimate of the spectrum in order 2, as well as an estimate of the uncontaminated flux of order 1 in the overlap region. However, the pipeline is not yet able to extract a spectrum using the GR700XD/F277W combination. Updates to the support this combination are in development.
calwebb_tso3
Data products from the TSO3 stage of the pipeline have not been fully verified. There are known differences between the extracted 1-D spectrum (*x1d.fits) from the calwebb_spec2 stage of the pipeline and the calwebb_tso3 stage of the pipeline. The step extract_1d is run in both the calwebb_spec2 and calwebb_tso3 stages of the pipeline which produces "*x1d.fits" files from each stage. In calwebb_spec2, the photometric correction (photom step) is applied after the extract_1d step due to the ATOCA algorithm used to disentangle the overlap in orders 1 and 2. In calwebb_tso3, the photom step will have been run before extract_1d which is the default for other observing modes but inappropriate for SOSS data.
Observers are encouraged to use data products from calwebb_spec2 for data analysis.
Summary of common issues and workarounds
The sections above provide detail on each of the known issues affecting NIRISS SOSS data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.
Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
---|---|---|---|---|
NR-SOSS05: Observations show a background that abruptly increases in flux at around column ~700. | This is due to the background (typically dominated by zodiacal light background at NIRISS/SOSS wavelengths) being dispersed by the GR700XD grism into the detector (see Albert et al., 2023; Section 8.5). | Scale the background taken during commissioning for SUBSTRIP256 and for SUBSTRIP96 to observations using non-illuminated regions of the detector. | N/A | Created issue Calibration program 4479 is studying background templates at different positions in the sky, as well as the variation of the background due to the pupil wheel position (PWCPOS) varying from visit to visit. Implementation on the JWST pipeline is still under development. |
NR-SOSS06: There is evident banding in the cross-dispersion direction, which varies from group-to-group and integration-to-integration. | This is due to residual, uncorrected 1/f noise left over from the reference pixel correction step. | There are several community tools available designed to remove 1/f noise. | N/A | Created issue A mitigation plan is under development. |
NR-SOSS07: Large number of outliers identified in the jump detection step, which cause lower SNR in the "_rateints" (count rate per integration) products. | The default parameters of the jump step in the JWST pipeline are too aggressive and flag too many pixels as cosmic rays. | Re-run the jump detection step in calwebb_detector1 with a rejection_threshold value > 10 (default is 4.0). | N/A | Created issue A mitigation plan is under development. |
NR-SOSS08: The pipeline is unable to extract a SOSS spectrum with the GR700XD/F277W combination. | The pipeline currently does not support this grism/filter combination. | There is currently not a workaround. | N/A | Updated issue As of pipeline build 11.0, the pipeline no longer throws an error when processing SOSS data using the GR700XD/F277W combination and instead exits gracefully at the end of the level 2a stage. Updates to the pipeline code to support the GR700XD/F277W combination are under development. |
NR-SOSS01: Too many pixels are flagged as "DO_NOT_USE" which are then set to "NaN" values in the science array. Some of these pixels are fine for SOSS data analysis. | Several detector-level reference files (mask, linearity, saturation, super bias) need to be updated to not cause certain pixels to be flagged as "DO_NOT_USE". | Use the attached Jupyter notebook to turn off setting pixels to "DO_NOT_USE". (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated Operations Pipeline New reference files were delivered to CRDS which no longer erroneously flag good pixels as "DO_NOT_USE". The new linearity and saturation reference files are available in CRDS contexts "jwst_1152.pmap" and later. The new subarray superbias reference files are available in CRDS contexts "jwst_1170.pmap" and later. Affected data products will be reprocessed; reprocessing of affected data typically takes 2–4 weeks. | |
NR-SOSS02: The shape of the spectral trace varies from visit to visit for all the NIRISS/SOSS orders. Distortion is higher at longer wavelengths. This might lead to lower SNR in the extracted spectra. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the shape of the spectral trace. | Use the PASTASOSS package to predict spectral traces given the PWCPOS, or use the package as a starting point to perform spectral tracing and thus extraction. | Updated issue Fixed in JWST pipeline build 11.1.1. | |
NR-SOSS03: The wavelength solution varies from visit to visit, appearing to be shifted by a handful of pixels. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the wavelength solution. | Use the PASTASOSS package to predict the wavelength solution for the visit. | Updated issue Fixed in JWST pipeline build 11.1. | |
NR-SOSS04: The shape of the spectral trace and wavelength solution sometimes varies from exposure to exposure on a given visit, including between subsequent CLEAR and F277W exposures. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position. When the position lands above a certain threshold from the commanded position, the onboard mechanism tries to re-adjust the position between exposures until it meets the threshold. | If trying to match CLEAR and F227W exposures, perform a translation and/or rotation of the F227W exposure to match the CLEAR exposure. Use stellar lines to match the wavelength solution on the F277W exposure. | N/A | Updated Operations Pipeline The NIRISS team has implemented the addition of a "Short First Exposure" consisting of a single group, single integration at the beginning of every exposure that requires an F277W exposure. This will give the on-board mechanism the opportunity to re-adjust the pupil wheel position if needed before the science exposures. This was implemented in the NIRISS SOSS template in APT 2023.5 (release date: August 24, 2023). |
References
Baines, T., et al. 2023a, JWST-STScI-008448
Characterization of the visit-to-visit Stability of the GR700XD Spectral Traces for NIRISS/SOSS Observations
Baines, T., et al. 2023b, JWST-STScI-008571
Characterization of the visit-to-visit Stability of the GR700XD Wavelength Calibration for NIRISS/SOSS Observations
Darveau-Bernier, A., et al. 2022, PASP, 134, 094502
ATOCA: an Algorithm to Treat Order Contamination. Application to the NIRISS SOSS Mode
ADS
Martel, A., 2022, JWST-STScI-008298
The Early Behavior of the NIRISS Pupil Wheel and Filter Wheel
McCullough, P. R., et al. 2008, PASP, 120, 759
Quantum Efficiency and Quantum Yield of an HgCdTe Infrared Sensor Array
ADS
Radica, M., et al., 2023, MNRAS, vol. 524, issue 1
Awesome SOSS: transmission spectroscopy of WASP-96b with NIRISS/SOSS
ADS
Rauscher, B. J., et al., 2014, PASP, vol. 126, issue 942
New and Better Detectors for the JWST Near-Infrared Spectrograph
ADS
Links
Predicting Accurate Spectra Traces in Astrophysical SOSS Spectra (PASTASOSS): https://github.com/spacetelescope/pastasoss