JWST NIRSpec IFU Pipeline Caveats

Some JWST Science Calibration Pipeline features specific to NIRSpec IFU data are described in this article, providing an overview of known issues that users should be aware of for their use of this science mode.

On this page

Summary of specific NIRSpec IFU pipeline issues

The information in this table about NIRSpec IFU pipeline issues is excerpted from Known Issues with JWST Data Products. 

SymptomsCauseWorkaroundMitigation Plan
NS-IFU03: Negative and/or surplus flux in the extracted 1-D spectra is seen, typically with an irregular wavelength-dependent undulation.

Correlated noise from low-level detector thermal instabilities, seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector. While the IRS2 readout modes reduces this effect, it is not completely eliminated.

Run the NSClean script developed by B. Rauscher on count rate images, using an appropriate mask.

Created issue

There will be an eventual inclusion of a cleaning algorithm in the Science Calibration Pipeline, pending further testing (timeline uncertain, but possibly in the updated operations pipeline planned for installation in November 2023).

NS-IFU04: There is missing flux in the "x1d" spectrum of point sources.

Astrometric or pointing uncertainty may cause the default extraction aperture to miss the intended target or be off-center.

Option 1: Rerun the extract_1d pipeline step in calwebb_spec3 at specified coordinates.

Option 2: Install the latest release of the jwst package and then run the Science Calibration Pipeline on the affected dataset. Starting in jwst 1.11.0, the extract_1d step supports setting ifu_autocen = True.

Created issue

An IFU astrometric solution was updated in July 2023, but small pointing offsets may remain. Reprocessing of affected data typically takes 2–4 weeks after the update.

NS-IFU05: Spectra extracted from single spaxels on/near point sources show a sinusoidal modulation.

NIRSpec is undersampled, and distortion causes spectral traces (particularly for the gratings) to be curved on the detector. Resampling the raw data to a rectified data cube introduces artifacts if extracting spectra on scales smaller than the PSF.  See detailed discussion by Law et al. 2023.

Extract spectra from larger apertures comparable in width to the PSF. Combining dithers/nods also reduces, but does not eliminate, the effect.

Created issue

A mitigation plan is under investigation.

NS-IFU06: The centroid of point sources appears to drift slightly as a function of wavelength.

The cause is unclear, but likely related to the filter transmission. The drift is typically of order 20 milliarcsec over the wavelength range of a given disperser.

None.

Updated issue

None at this time. Any action regarding pipeline or post-processing mitigation await an investigation on the root cause, which has just begun.

NS-IFU01: There is a hole at the peak of the PSF or an otherwise distorted PSF in IFU "s3d" cube.This is due to overly aggressive outlier detection.

Option 1: Turn off outlier rejection in the cube_build step of calwebb_spec3. However, this may allow other outliers to remain in the cube.

Option 2: Reprocess data using the  Science Calibration Pipeline, jwst 1.11.3 and onward. This version will be installed in the operations pipeline with build 9.3), planned for installation on .

Updated operations pipeline

Update outlier detection algorithm. STScI will reprocess affected data products with an updated operations pipeline, planned for installation on . Reprocessing of affected data typically takes 2–4 weeks after the update.


NS-IFU02: Flux is not conserved in 1-D-extracted spectra of point sources when using a different output spaxel sampling.

The cube_build algorithm was designed to conserve flux assuming input units of surface brightness. However, the NIRSpec point source calibration produces units of flux density, which is not compatible.

If using data processed with the current operations pipeline build, do the following:

  1. change the header keyword SRCTYAPT in the primary extension of each "rate.fits" file to EXTENDED
  2. re-run the calwebb_spec2 pipeline to apply the surface brightness calibration
  3. change the header keyword SRCTYPE in the "SCI" extension header of the new "s3d" products to POINT
  4. re-run the extract_1d step

Otherwise, install the release candidate for the coming operations pipeline (jwst 1.11.3 and onward) and re-run calwebb_spec2.

Updated operations pipeline

The point source calibration for IFU data will be changed to surface brightness units, and 1-D spectra converted back to flux density. STSCI will reprocess affected data products with an updated operations pipeline, planned for installation on . Reprocessing of affected data typically takes 2–4 weeks after the update.


Cube building

ReadTheDocs (External Link): Cube Building

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

The cube_build step of the pipeline combines the individual 2-D IFU slice images and creates a 3-D spectral cube. The IFU cubes are, by default, constructed with north pointing up and east to the left. Cube building can be done using either the default 3-D drizzle algorithm, or alternatively, the Shepard's method of weighting. The current pipeline default setting is the 3-D drizzle algorithm.

  • To use 3-D drizzle, set WEIGHTING = DRIZZLE
  • To use Shepard's method with exponential or linear weighting, set WEIGHTING = emsm or msm  

It is sometimes useful to build a cube in the detector frame (for example, when analyzing the point spread function), rather than in sky coordinates. To build the cube in detector coordinates:

Set coord_system = ifualign

Cube building artifacts

There is spatial undersampling in the IFU that may result in an apparent "ringing" in the spectrum upon resampling during cube building. This is inherent to the cube building process and there is currently no correction in the pipeline for it. Ways to mitigate this effect are currently being investigated. It may help to use a larger spatial extraction region to reduce the amplitude of the effect in extracted 1-D spectra.

Flux Non-Conservation

The cube build step in pipeline build 9.2 and earlier have a bug that caused the flux to not be conserved in the case of point sources, which resulted in flux densities being too high by ~10–15% for the default cube spaxel sampling. A fix is now available in pipeline build 9.3.

Outlier Detection

The outlier detection algorithm has 2 issues that can result in incorrect flagging of outliers or missed outliers:

  1. The algorithm can be overly aggressive in flagging outliers between dithers since the PSF is undersampled. This can result a  donut-like PSF for point source data, where peak pixels are incorrectly flagged as outliers.
  2. The median cube used to detect outliers is not optimally sampled and leaves too many true outliers undetected.

These issues with outlier detection are mitigated in pipeline build 9.3. It may be necessary to re-run the outlier detection step of the pipeline to improve outlier detection for data processed with older versions of the pipeline.

Correlated 1/f read noise

Correlated read noise, also known as 1/f noise, from detector thermal instabilities can lead to vertical banding and and a "picture frame" effect. While this effect is reduced by use of the IRS2 readout pattern, residual 1/f noise that is not corrected by the IRS2 reference pixel correction requires further processing to remove. This most prominently affects the NRS2 detector, and can result in apparent continuum offsets and/or spurious structure for read noise-limited data, with apparent flux discrepancies of up to 30%. 

The NSClean algorithm may be utilized to remove most of this effect, and is being considered for eventual implementation in the pipeline.

Background subtraction

ReadTheDocs (External Link): Background Subtraction

Background subtraction is automatically applied by the calwebb_spec3 pipeline for nodded observations or observations with dedicated background or leakage observations. It is not automatically applied for observations that have off-scene background observations that were not linked to the target in APT. Custom background subtraction may be required depending on science use case. In particular, a 1-D master background spectrum may be specified when running the calwebb_spec3 pipeline.

Leakage of flux through the MSA may be significant in the case of bright extended targets or point sources in stuck-open shutters. If dedicated leakage observations were obtained at every dither or nod, the pipeline will use them to subtract the leakage signal. However, if leakage observations were only acquired at one dither or nod, the pipeline may not process the data correctly. In this case, custom background subtraction may be necessary.

Default spectral extraction location

The default 1-D extraction aperture centering for point sources is based on source coordinates, but often is not perfectly centered on the source because of uncertainties in the pointing information. Users can specify the desired location for pipeline extraction on the command line with the parameter "center_xy".



Latest updates


Originally published