NIRSpec FS Known Issues
Known issues specific to NIRSpec fixed slit data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science.
On this page
Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.
Please also refer to NIRSpec Fixed Slit Calibration Status for an overview of the current photometric and wavelength calibration accuracy of NIRSpec fixed slit data products.
Artifacts
Information on NIRSpec instrument artifacts are found on the main NIRSpec Known Issues page.
Pipeline notes
Correlated 1/f read noise
The effects of 1/f noise for NIRSpec FS are shown in the FS 1/f noise workaround notebook, which also demonstrates the use of the NSClean algorithm to remove most of this effect. NSClean is now implemented in the pipeline (v1.13.4 onwards) as a non-default option. Further details on how to invoke NSClean within the science calibration pipeline and adjust default parameters are described in the 1/f noise workaround notebook.
Resampling of 2-D spectra
Stages 2 and 3 of the pipeline resample the 2-D spectra before doing the final 1-D extraction. The resampling of 2-D spectra has been found to introduce extra artificial noise that is reflected in the 1-D spectra, and significantly decreases the signal-to-noise. This issue is mitigated by setting the parameter weight type to be exptime in the resample step in stage 2 and 3 of the pipeline.
Flux in off-center dithers/nods
In some cases, the flux of the spectra obtained at the off-center dithers/nod positions is higher by a few percent from the flux at the center center nod. This issue is under investigation.
Summary of common issues and workarounds
The sections above provide details on each of the known issues affecting NIRSpec fixed slit data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-FS05: Many significant outliers and "NaN" appear in the 2-D and 1-D extracted spectra. | The outlier_detection step generally has a hard time finding many outliers. Step parameters need to be tuned to the noise characteristics of each data set, although in many cases outliers are still missed. Also, the initial bad pixel flags set in the mask and dark reference files for subarrays contain too many "DO_NOT_USE" designations, which result in "NaN" pixel values. | "NaN" values can be resolved by removing "DO_NOT_USE" DQ flags and re-running the ramp_fit step in calwebb_detector1. A worked example on how to do that, using NIRISS SOSS data, is available. For outlier improvement, rerun the outlier_detection step in calwebb_spec3 with different values of the snr parameter. | N/A | Updated issue Reference files were updated in October 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. In the longer term, updated algorithms are under investigation, possibly for inclusion in the Operations Pipeline update planned for February 2024. |
NS-FS07: Lower than expected S/N and/or larger than expected discrepancies between dither positions. | Use of the default “inverse variance weighting” using read noise variance in the resample_spec step (resample_spec.weight_type = ivm) does not appear to be appropriate for high signal-to-noise data | When running calwebb_spec2, set resample_spec.weight_type = exptime, and when running calwebb_spec3, set spec3.weight_type = exptime. | N/A | Created issue Investigate whether to change the default weighting to exptime using a modified parameter reference file, and/or investigate other algorithms that handle the high S/N limit more gracefully. Also consider extraction and image combination algorithms that don’t require resampling the 2-D data. |
NS-FS08: Lower than expected flux and/or flux inconsistencies between closely spaced dither positions that may result in issues with outlier rejection. In extreme cases this may result in an ~20% error in the absolute flux calibration, although more commonly, the effect is at the ~5% level. | When closely spaced nod positions are used as background members for each other, some source flux is subtracted from the extraction region along with the background, but the fraction of source flux subtracted varies with the dither separation of each background member. Currently the pipeline does not account for this in the flux calibration. | For sufficiently bright sources, consider omitting background members in the level 2 association, and/or find separate background exposures from other observations. Otherwise, when using 5 primary dither points, especially in the S1600A1, edit level 2 association to remove background members separated by less than about 0.375” from the science member in that association. Note that the pipeline already knows not to use sub-pixel spatial or dispersion direction dithers as background members for other exposures at the same primary dither position. | N/A | Created issue Modify default pipeline rules to enforce a minimum separation for background members used in calwebb_spec2. In the longer run, investigate adding a step to calwebb_spec2 to estimate the flux loss due to subtraction of other dithers and correct the flux in the "cal" files to account for this. Once fixes are in place to minimize these effects, reconsider all F-flat reference files for FS, (and possibly also MOS), to allow improved absolute and relative flux accuracy. |
NS-FS09: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture. | The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture. | Use only the spectra extracted from the upper dither positions in the affected wavelength range. | N/A | Created issue Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE. |
| NS-FS03: Spectra obtained with the SLIT = S200A1 and S200A2 option in APT are not combined to create a single spectrum with continuous wavelength coverage. | Association logic does not realize that SOURCEID = 1 observed with S200A1 is the same target as SOURCEID = 2 observed with S200A2. | Update | updated Operations Pipeline Enhanced the association logic and changed the | |
| NS-FS06: Level 3 extracted spectra have errors that are all "NaN." | The flat field reference file uncertainties are currently zero. The resultant flat error component calculated from these is "NaN", which propagates to the combined error as "NaN". | Recalculate the combined error using only the read noise and photon noise components; instructions will soon be available. See this worked example for more on how to do this. (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated issue, new due date for reference file Reprocess data with an enhanced calibration reference file (flat) in CRDS. An update is planned for early 2024. Reprocessing of old data typically takes 2–4 weeks after the update. | |
| NS-FS04: FS spectra obtained with the G140M grating exhibit flux discrepancies of up to 20%. | No spectrophotometric observations for this grating were obtained during commissioning, so the F-flat reference file (flux calibration) for this specific case is the model-based pre-flight version. | None. | Issue resolved Spectrophotometric observations with this configuration were obtained in August, and updated F-flat reference file were delivered on August 24, 2023. | |
| NS-FS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. | Some subarrays have no reference pixels, which means bias drifts are not corrected. | Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps. | Updated Operations Pipeline The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. STScI reprocessed the affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-FS02: Background spectra taken from one fixed slit does not match the point source spectrum extracted from a different slit. | The pathloss step as applied to FS data had a bug that led to the correction being applied twice. This affects all FS exposures; the impact on point sources is minor (a few percent at most), but higher for extended sources. | None. | Updated Operations Pipeline The pathloss correction was applied only once. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) |