NIRSpec FS Known Issues

Known issues specific to NIRSpec fixed slit data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science.

On this page


Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRSpec Fixed Slit Calibration Status for an overview of the current photometric and wavelength calibration accuracy of NIRSpec fixed slit data products.


Artifacts

Information on NIRSpec instrument artifacts are found on the main NIRSpec Known Issues page.


Pipeline notes

Correlated 1/f read noise

The effects of 1/f noise for NIRSpec FS are shown in the FS 1/f noise workaround notebook, which also demonstrates the use of the NSClean algorithm to remove most of this effect. NSClean is now implemented in the pipeline (v1.13.4 onwards) as a non-default option. Further details on how to invoke NSClean within the science calibration pipeline and adjust default parameters are described in the 1/f noise workaround notebook. 

Resampling of 2-D spectra

Stages 2 and 3 of the pipeline resample the 2-D spectra before doing the final 1-D extraction. Currently the resample step in stage 2 and 3 of the pipeline resamples 2-D spectra with a parameter weight_type = exptime.  The alternative to this is using weight_type = ivm, which weights each pixel by its inverse readnoise variance.  While the ivm weighting can reduce the number of outliers that are retained through resampling, this method has been found to produce extra artificial noise in some 1-D spectra, particularly for high S/N spectra, and significantly decreases their signal-to-noise.  An intermediate resampling is also performed for the outlier_detection step in stage 3 of the pipeline to identify outliers in dithered spectra, and uses the ivm weighting in order to detect these outliers. 

Flux in off-center dithers/nods

In some cases, the flux of the spectra obtained at the off-center dithers/nod positions is higher by a few percent from the flux at the center center nod. This issue is under investigation.


Summary of common issues and workarounds

The sections above provide details on each of the known issues affecting NIRSpec fixed slit data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.


SymptomsCauseWorkaroundFix buildMitigation Plan

NS-FS08: Lower than expected flux and/or flux inconsistencies between closely spaced dither positions. This may result in up to a ~5% error in the absolute flux calibration.

When closely spaced nod positions are used for nod subtracted backgrounds, some source flux may be subtracted from the extraction region depending on dither position and dither pattern. Currently the pipeline does not account for this in the flux calibration.

For sufficiently bright sources, consider omitting background members in the level 2 association, and/or perform background subtraction during 1D extraction. 

 N/A

Updated issue

As of jwst 1.16.0 (and later versions) S1600A1 5-point nod associations do not include neighboring nods as background members, addressing the worst case of nod oversubtraction.

Future improvements are pending an investigation into how to best correct nod-over subtraction more generally.

NS-FS09: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture.

The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture.

Use only the spectra extracted from the upper dither positions in the affected wavelength range.

 N/A

Created issue

Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE.

NS-FS05: Many significant outliers appear in the 2-D and 1-D extracted spectra.

The outlier_detection step generally has a hard time finding many outliers. Step parameters need to be tuned to the noise characteristics of each data set, although in many cases outliers are still missed. 

For outlier improvement, rerun the outlier_detection step in calwebb_spec3 using jwst 1.17.0 (or later versions).

11.2

Issue resolved

The outlier_detection step in calwebb_spec3 has been improved in jwst 1.17.0 (and later versions) and is set to be included in the Operations Pipeline update planned for March 2025.  Reprocessing of affected data typically takes 2–4 weeks after the update.

Any users reprocessing their data should use jwst 1.17.0 (or later versions) to include these improvements.

NS-FS11:  Level 3 spectra  that fall on NRS1 and NRS2 (H-grating data) show unexpected outliers towards the blue end of NRS2 data.

Because the read noise variances differ between NRS1 and NRS2, when using the default read noise variance weighting, outlier_detection.weight_type = ivm, and outlier_detection.maskpt = 0.7, the outlier_detection step may mask parts of the observed spectra on NRS2, producing unexpectedly noisy spectra in the masked regions.

Rerun calwebb_spec3 with the improved outlier_detection step and updated default parameters in jwst 1.17.0 (or later versions).

Default outlier_detection step parameters have been set to outlier_detection.weight_type = ivm and outlier_detection.maskpt = 0.0 as of CRDS context jwst_1312.pmap (used automatically by jwst 1.17.0 and later).

11.2

Issue resolved

The outlier_detection step in calwebb_spec3 has been improved in jwst 1.17.0 (and later versions) and is set to be included in the Operations Pipeline update planned for March 2025.  Reprocessing of affected data typically takes 2–4 weeks after the update.

Any users reprocessing their data should use jwst 1.17.0 (or later versions) to include these improvements.

SymptomsCauseWorkaroundFix BuildMitigation Plan
NS-FS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more.

Some subarrays have no reference pixels, which means bias drifts are not corrected.

Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps.

 9.3

Updated Operations Pipeline

The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. STScI reprocessed the affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.)

NS-FS02: Background spectra taken from one fixed slit does not match the point source spectrum extracted from a different slit.

The pathloss step as applied to FS data had a bug that led to the correction being applied twice.

This affects all FS exposures; the impact on point sources is minor (a few percent at most), but higher for extended sources.

None.

9.3

Updated Operations Pipeline

The pathloss correction was applied only once. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.)

NS-FS03: Spectra obtained with the SLIT = S200A1 and S200A2 option in APT are not combined to create a single spectrum with continuous wavelength coverage.Association logic does not realize that SOURCEID = 1 observed with S200A1 is the same target as SOURCEID = 2 observed with S200A2.

Update SOURCEID in the "SCI" extension headers, so that a single unique value is used for all exposures of the science target. Manually create an association file that includes all exposures of the science target as members. Use calwebb_spec3 to reprocess the association. See the workaround notebook for an example (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline).

10.0

updated Operations Pipeline

Enhanced the association logic and  changed the SOURCEID numbering scheme. STScI is reprocessing affected data products in the Operations Pipeline that was updated in December 2023. Reprocessing of affected data typically takes 2–4 weeks after the update.

NS-FS04: FS spectra obtained with the G140M grating exhibit flux discrepancies of up to 20%.

No spectrophotometric observations for this grating were obtained during commissioning, so the F-flat reference file (flux calibration) for this specific case is the model-based pre-flight version.

None.

9.3

Issue resolved

Spectrophotometric observations with this configuration were obtained in August, and updated F-flat reference file were delivered on August 24, 2023.

NS-FS06: Level 3 extracted spectra have errors that are all "NaN."

The flat field reference file uncertainties are currently zero. The resultant flat error component calculated from these is "NaN", which propagates to the combined error as "NaN".

Recalculate the combined error using only the read noise and photon noise components; instructions will soon be available. See this worked example for more on how to do this.  (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline).

10.0

Updated issue, new due date for reference file

Reprocess data with an enhanced calibration reference file (flat) in CRDS. An update is planned for early 2024. Reprocessing of old data typically takes 2–4 weeks after the update.

NS-FS07: Lower than expected S/N and/or larger than expected discrepancies between dither positions.

Use of the default “inverse variance weighting” using read noise variance in the resample_spec step (resample_spec.weight_type = ivm) does not appear to be appropriate for high signal-to-noise data

When running calwebb_spec2, set resample_spec.weight_type = exptime, and when running calwebb_spec3, set spec3.weight_type = exptime.

 11.0

Issue resolved

The default resampling weight has been set to exptime as of JWST CRDS context jwst_1252.pmap (and later), which is included in the Operations Pipeline update on August, 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update.  This improves some, but not all, discrepancies between dither positions.

In the longer term consider extraction and image combination algorithms that don’t require resampling the 2-D data, or extraction options with improved centroiding.

NS-FS10: Discrepancies in extracted 1D fluxes at the 3-5% level when changing the assign_wcs step slit width.

There is a bug in the NIRSpec spectral resampling that causes the output pixel area to differ from the input pixel area depending on the assign_wcs slit width.

Reprocess spectra using the new version of the calibration pipeline software (jwst 1.16.0 or later and using JWST CRDS context jwst_1298.pmap or later).  The spectral resampling now conserves flux regardless of slitlet length.

 11.1

Issue resolved

The spectral resampling has been improved in jwst 1.16.0 (and later versions) and is set to be included in the Operations Pipeline update in January 2025.  Reprocessing of affected data typically takes 2–4 weeks after the update.  This improves some, but not all, discrepancies between dither positions.  

Any users reprocessing data with jwst 1.16.0 (and later versions) should use JWST CRDS context jwst_1298.pmap (or later), which includes updated flux calibrations for the new spectral resampling code.


Notable updates
  •  
    • Moved retired issues NS-FS01, NS-FS02, NS-FS03, NS-FS04, NS-FS06, NS-FS07, NS-FS10 to a separate expandable table. 
    • Updated issue NS-FS08 mitigation plan. 
    • Resolved issues NS-FS05, NS-FS11 with final mitigation plan.

  •  

    Updated mitigation plan for issue NS-FS05.
    Created issue NS-FS11: Level 3 spectra  that fall on NRS1 and NRS2 (H-grating data) show unexpected outliers towards the blue end of NRS2 data.
    Updated text in section Resampling of 2D spectra for default parameter values.

  •  
    Updated mitigation plan for issues NS-FS07, NS-FS10


  • Updated issues NS-FS05, NS-FS07, NS-FS08
    Added issue NS-FS10: Discrepancies in extracted 1D fluxes at the 3-5% level when changing the assign_wcs step slit width.
Originally published