NIRSpec FS Known Issues
Known issues specific to NIRSpec fixed slit data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science.
On this page
Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.
Please also refer to NIRSpec Fixed Slit Calibration Status for an overview of the current photometric and wavelength calibration accuracy of NIRSpec fixed slit data products.
Artifacts
Information on NIRSpec instrument artifacts are found on the main NIRSpec Known Issues page.
Pipeline notes
Correlated 1/f read noise
The effects of 1/f noise for NIRSpec FS are shown in the FS 1/f noise cleaning notebook, which also demonstrates the use of the NSClean algorithm to remove most of this effect. NSClean is implemented in the pipeline versions from v1.13.4 through v1.20.2 as a non-default option. As of JWST pipeline v2.0.0 (and later versions) there are multiple 1/f cleaning options in the pipeline provided by the optional clean_flicker_noise step available in both calwebb_detector1 and calwebb_spec2. And, as of JWST pipeline v3.0.0 the clean_flicker_noise step is run by default in calwebb_spec2 for FS data. In some cases, additional improvements can be gained by also running clean_flicker_noise step available in calwebb_detector1; however, in other cases cleaning 1/f noise at this stage can leave striping artifacts in the resulting rate images. The FS pipeline notebook shows how the clean_flicker_noise step can be run as part of pipeline processing.
Resampling of 2-D spectra
Stages 2 and 3 of the pipeline resample the 2-D spectra before doing the final 1-D extraction. Currently the resample step in stage 2 and 3 of the pipeline resamples 2-D spectra with a parameter weight_type = exptime. The alternative to this is using weight_type = ivm, which weights each pixel by its inverse readnoise variance. While the ivm weighting can reduce the number of outliers that are retained through resampling, this method has been found to produce extra artificial noise in some 1-D spectra, particularly for high S/N spectra, and significantly decreases their signal-to-noise. An intermediate resampling is also performed for the outlier_detection step in stage 3 of the pipeline to identify outliers in dithered spectra, and uses the ivm weighting in order to detect these outliers.
Flux in off-center dithers/nods
In some cases, the flux of the spectra obtained at the off-center dithers/nod positions is higher by a few percent from the flux at the center center nod. This issue is under investigation.
Summary of common issues and workarounds
The sections above provide details on each of the known issues affecting NIRSpec fixed slit data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
NS-FS08: Lower than expected flux and/or flux inconsistencies between closely spaced dither positions. This may result in up to a ~5% error in the absolute flux calibration. | When closely spaced nod positions are used for nod subtracted backgrounds, some source flux may be subtracted from the extraction region depending on dither position and dither pattern. Currently the pipeline does not account for this in the flux calibration. | For sufficiently bright sources, consider omitting background members in the level 2 association, and/or perform background subtraction during 1D extraction. | N/A | Updated issue As of jwst 1.16.0 (and later versions) S1600A1 5-point nod associations do not include neighboring nods as background members, addressing the worst case of nod oversubtraction. Future improvements are pending an investigation into how to best correct nod-over subtraction more generally. |
NS-FS09: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture. | The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture. | Use only the spectra extracted from the upper dither positions in the affected wavelength range. | N/A | Created issue Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE. |
Retired Issues:
| Symptoms | Cause | Workaround | Fix Build | Mitigation Plan |
|---|---|---|---|---|
| NS-FS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. | Some subarrays have no reference pixels, which means bias drifts are not corrected. | Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps. | Updated Operations Pipeline The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. STScI reprocessed the affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-FS02: Background spectra taken from one fixed slit does not match the point source spectrum extracted from a different slit. | The pathloss step as applied to FS data had a bug that led to the correction being applied twice. This affects all FS exposures; the impact on point sources is minor (a few percent at most), but higher for extended sources. | None. | Updated Operations Pipeline The pathloss correction was applied only once. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-FS03: Spectra obtained with the SLIT = S200A1 and S200A2 option in APT are not combined to create a single spectrum with continuous wavelength coverage. | Association logic does not realize that SOURCEID = 1 observed with S200A1 is the same target as SOURCEID = 2 observed with S200A2. | Update | updated Operations Pipeline Enhanced the association logic and changed the | |
| NS-FS04: FS spectra obtained with the G140M grating exhibit flux discrepancies of up to 20%. | No spectrophotometric observations for this grating were obtained during commissioning, so the F-flat reference file (flux calibration) for this specific case is the model-based pre-flight version. | None. | Issue resolved Spectrophotometric observations with this configuration were obtained in August, and updated F-flat reference file were delivered on August 24, 2023. | |
| NS-FS05: Many significant outliers appear in the 2-D and 1-D extracted spectra. | The outlier_detection step generally has a hard time finding many outliers. Step parameters need to be tuned to the noise characteristics of each data set, although in many cases outliers are still missed. | For outlier improvement, rerun the outlier_detection step in calwebb_spec3 using jwst 1.17.0 (or later versions). | Issue resolved The outlier_detection step in calwebb_spec3 has been improved in jwst 1.17.0 (and later versions) and is set to be included in the Operations Pipeline update planned for March 2025. Reprocessing of affected data typically takes 2–4 weeks after the update. Any users reprocessing their data should use jwst 1.17.0 (or later versions) to include these improvements. | |
| NS-FS06: Level 3 extracted spectra have errors that are all "NaN." | The flat field reference file uncertainties are currently zero. The resultant flat error component calculated from these is "NaN", which propagates to the combined error as "NaN". | Recalculate the combined error using only the read noise and photon noise components; instructions will soon be available. See this worked example for more on how to do this. (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated issue, new due date for reference file Reprocess data with an enhanced calibration reference file (flat) in CRDS. An update is planned for early 2024. Reprocessing of old data typically takes 2–4 weeks after the update. | |
NS-FS07: Lower than expected S/N and/or larger than expected discrepancies between dither positions. | Use of the default “inverse variance weighting” using read noise variance in the resample_spec step (resample_spec.weight_type = ivm) does not appear to be appropriate for high signal-to-noise data | When running calwebb_spec2, set resample_spec.weight_type = exptime, and when running calwebb_spec3, set spec3.weight_type = exptime. | Issue resolved The default resampling weight has been set to exptime as of JWST CRDS context jwst_1252.pmap (and later), which is included in the Operations Pipeline update on August, 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. In the longer term consider extraction and image combination algorithms that don’t require resampling the 2-D data, or extraction options with improved centroiding. | |
NS-FS10: Discrepancies in extracted 1D fluxes at the 3-5% level when changing the assign_wcs step slit width. | There is a bug in the NIRSpec spectral resampling that causes the output pixel area to differ from the input pixel area depending on the assign_wcs slit width. | Reprocess spectra using the new version of the calibration pipeline software (jwst 1.16.0 or later and using JWST CRDS context jwst_1298.pmap or later). The spectral resampling now conserves flux regardless of slitlet length. | Issue resolved The spectral resampling has been improved in jwst 1.16.0 (and later versions) and is set to be included in the Operations Pipeline update in January 2025. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. Any users reprocessing data with jwst 1.16.0 (and later versions) should use JWST CRDS context jwst_1298.pmap (or later), which includes updated flux calibrations for the new spectral resampling code. | |
NS-FS11: Level 3 spectra that fall on NRS1 and NRS2 (H-grating data) show unexpected outliers towards the blue end of NRS2 data. | Because the read noise variances differ between NRS1 and NRS2, when using the default read noise variance weighting, outlier_detection.weight_type = ivm, and outlier_detection.maskpt = 0.7, the outlier_detection step may mask parts of the observed spectra on NRS2, producing unexpectedly noisy spectra in the masked regions. | Rerun calwebb_spec3 with the improved outlier_detection step and updated default parameters in jwst 1.17.0 (or later versions). Default outlier_detection step parameters have been set to outlier_detection.weight_type = ivm and outlier_detection.maskpt = 0.0 as of CRDS context jwst_1312.pmap (used automatically by jwst 1.17.0 and later). | Issue resolved The outlier_detection step in calwebb_spec3 has been improved in jwst 1.17.0 (and later versions) and is set to be included in the Operations Pipeline update planned for March 2025. Reprocessing of affected data typically takes 2–4 weeks after the update. Any users reprocessing their data should use jwst 1.17.0 (or later versions) to include these improvements. |