- Home
- Known Issues with JWST Data
- Known Issues with JWST Data - High-Level Summary
Known Issues with JWST Data - High-Level Summary
All tables about common issues and workarounds for each JWST observing mode, from other pages, are reproduced here in a single location. They are simply mirrored information in those instrument and mode-specific pages; users are encouraged to consult the individual mode pages for further information as the mode pages generally contain significant additional information not captured within the tables.
Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.
On this page
Multi-instrument issues
General issues
The information in this table is excerpted from Known Issues with JWST Data.
| Symptoms | Cause | Workaround | Fix build | Mitigation plan |
|---|---|---|---|---|
| GI02: Embedded world coordinate system (WCS) in JWST data products is incorrect. | Errors in the guide star catalog, misidentified guide stars, and uncertainties in the spacecraft roll angle result in errors in the WCS of pipeline data products even when target acquisition was performed to place science targets in the correct location. Typical errors are a few tenths of an arcsec, with some cases that are greater than 1 arcsec. | The workaround depends on instrument and mode. Also see issue NC-I01. | N/A | Updated issue Improve accuracy of the guide star catalog. This is a long-term project. (Updated "Workaround" to mention NC-I01) |
| GI03: Data contain shower and snowball artifacts. | These are caused by large cosmic ray impacts. The calwebb_detector1 pipeline includes a snowball/shower correction. This correction is run by default for many observing modes, but is turned off in some others until testing indicates satisfactory performance in corner cases. | There is no workaround that works for all science cases. The correction is not recommended for NIRISS SOSS or AMI, MIRI coronagraphic data, and data with 1–4 groups. For general science cases, users can re-run the pipeline calwebb_detector1 with the jump step parameters set: | N/A | Updated issue Snowball/shower correction in the jump detection step of calwebb_detector1 has been implemented in the pipeline and enabled via delivery of new parameter reference files for each instrument, as they become available. As of Build 10.2 automated correction occurs for most NIR modes and MIRI imaging. See the section titled "Large Events (Snowballs and Showers)" in the JumpStep documentation, and the Shower and Snowball Artifacts article, for more information on the status of these corrections. |
| GI05: Stage 3 processing of large imaging mosaics can take a longer than the normal amount of time to process. | Pipeline memory usage can be extremely large when certain steps are passed in a large volume of data. This can cause slowdowns when available memory is exceeded and swap space is used instead, or crashes in extreme cases. | None. | Updated issue The tweakreg and resample steps (along with the rest of the pipeline) have been rewritten to ease memory pressure. Likewise, the resample step has a 'in_memory' argument that can be toggled to mitigate memory usage at the expense of runtime (and vice versa). Tests suggest significant performance improvement, although issues may still remain for large mosaics on machines with less than 64 GB of available RAM. If problems persist please file a JWST Help Desk so that developers can continue to investigate. | |
| GI04: NIR instruments only: There is large-scale striping due to 1/f noise (horizontal for NIRCam, vertical for NIRISS and NIRSpec) across the field and not fully removed in the reference pixel subtraction. Note that IRS2 readout for NIRSpec substantially mitigates this behavior. | 1/f noise from the SIDECAR ASICs (detector readout electronics) causes this effect. | There are several community tools available that are designed to remove 1/f noise.
| Updated issue to indicate that a general-purpose pipeline step clean_flicker_noise is now available for all NIR instruments. This step is currently off by default, but can be enabled for offline rereductions. Additional work continues to extend and improve the step, and to improve 1/f correction via other means in future builds. See details at 1/f Noise |
General issues for time-series observations
The information in this table is excerpted from Known Issues with JWST Data.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|
MIRI Issues
MIRI coronagraphic imaging issues
The information in this table is excerpted from MIRI Coronagraphy Known Issues
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
MR-CI02: Users will want to perform their own PSF subtraction with the data products. | PSF subtraction has too many free parameters to capture them in a one-size-fits-all automated processing pipeline. | The public "Coronagraphy_ExampleNB" repository contains a suite of 6 notebooks that will guide the user through reprocessing coronagraphic data through all steps of the pipeline, as well as for customizing parameters at each step. Users will find notebooks suitable for processing "uncal" files retrieved from MAST through stage 2 to yield background-subtracted photometrically-calibrated data products suitable for use in PSF subtraction. Older notebooks are still available in the "old_pipeline_demos" subfolder. | N/A | Updated issue The MIRI team is always considering the scope of what data products the calwebb_coron3 pipeline stage should provide. |
MR-CI03: Tweak stage 1 and 2 parameters to optimize data calibration. | Default parameters may not be optimal for all datasets. | spaceKLIP, provided by the ERS team, can run pipeline steps as well as perform highly tunable PSF subtraction on the data products. There are examples of different pipeline parameter values they have found helpful, which can be found in the MIRI config file under the "tests" folder. Pipeline stages 1 and 2 are shared with the MIRI imager, so users may look there for more examples. | N/A | Created issue None at this time. |
| MR-CI04: Current Lyot flat-fielding causes a sharp discontinuity underneath the coronagraphic mask. | A sharp edge in the Lyot coronagraph's flat-fielding reference file creates a sharp discontinuity underneath the coronagraphic mask, in stage 2 pipeline products. This can result in artifacts during certain image processing steps that are sensitive to abrupt changes. | For now, users should be aware that this sharp discontinuity is an artifact of the calibration pipeline and does not represent astrophysical information. They should take this into account during post-processing. | N/A | Created issue Mitigation strategies are under investigation. |
| MR-CI01: Absolute flux calibration is incorrect. | An error was made when computing the aperture correction for coronagraphic PSFs. | Multiply fluxes by a factor that will be available by the end of August, 2023. | 10.0 | Updated Operations Pipeline Flux calibration "photom" reference files were updated in October. STScI will reprocess affected data products with updated calibration reference data. Reprocessing of affected data typically takes 2–4 weeks after the update. |
MIRI imaging issues
The information in this table is excerpted from MIRI Imaging Known Issues
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
MR-I02: The "cal" data product has "tree ring" like pattern. | The "tree ring" pattern of curves in MIRI images is a residual detector effect that is not removed in all cases by the reference files. | Creating and subtracting a median background can remove residual detector patterns in the background like the 'tree rings'. This notebook demonstrates how it can be done. | N/A | Created issue Reference file and pipeline updates may be available at a future date to remove residual patterns such as the "tree rings" from all data, but there is no timeline on when this might be accomplished. |
MR-I01: Image has residual background. | The backgrounds for longer wavelength images can be quite high, and difficult to work with if dedicated background images are not taken and subtracted off in the pipeline. | Users can create a median sky (background image) and subtract it off for themselves if there is no extended source such as a galaxy or nebula in the image. This is most useful for long wavelength imaging, but can also be used for shorter wavelength images. This notebook demonstrates how it can be done. | N/A | Created issue This is not something that can (or should) be changed in the pipeline, and will need to be done as an offline step as shown in the notebook. |
MR-I03: The photon count rate and derived flux are lower than predicted at wavelengths between 12.8 and 25.5 µm, with the effect increasing with wavelength (see JWST Observer). | The MIRI imager sensitivity at long wavelengths is decreasing with time. The root cause of this issue is still under investigation. Regular monitoring observations are being taken with the MIRI imager to measure the photometric response and to characterize the temporal trend. | New "photom" files were delivered on September 15, 2023 and there are 2 possible workarounds:
| Updated issue In the short term, new "photom" reference files tailored for specific epochs have been provided in CRDS. STScI will update the pipeline to include a new time-dependent throughput correction, expected to occur in February 2024. Both mitigation strategies will require a reprocessing of the data. |
MIRI low resolution spectroscopy (LRS) issues
The information in this table is excerpted from MIRI LRS Known Issues
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
MR-LRS03: Spectra of bright sources can experience data dropouts at wavelengths where they are brightest (in the "x1d" files). The problem can also be seen in 2-D spectral images, with the peak column flagged as "DO_NOT_USE" in several adjacent rows where the spectrum is brightest. | The jump detection step in calwebb_detector1 can flag pixels with strong signals as "DO_NOT_USE" in the spectral images even though the data are viable. These pixels are "NaN" in the spectral images and their signal is missing from the affected wavelengths in the extracted spectra. | Users can run calwebb_detector1 themselves and adjust the rejection_threshold parameter in the jump step to eliminate (or reduce) improper flagging. A generic workaround is challenging to produce as the occurrence and severity of the issue depends on several factors, e.g., target brightness and number of groups per integration. | N/A | Updated issue The jump step algorithm and default parameters are continually being examined and optimized; improvements are expected in future builds (spring 2024 and beyond). The issue likely cannot be entirely eliminated as it differs greatly between observations (and indeed many are not affected). |
| MR-LRS04: Spectra extracted from the LRS slit ("x1d" files) can show major disparities between the nods or with expected values. | The pathloss correction is generating incorrect results due to a poor understanding of the location of the source in the slit. | The "pathloss" reference file has already been modified to apply a correction as though the source were in the center of the slit, no matter where it actually is (or where it is believed to be). Effective with build 10.0, even if the pipeline thinks a source is out of the slit, it will still be corrected as though it were in the center of the slit. | N/A | Created issue The Science Calibration Pipeline will be modified to allow the user to rerun the pathloss correction with a default position of the center of the slit. The Operations Pipeline will be modified to determine the source position based on the TA verification image. Neither of these solutions are ready for implementation. |
| MR-LRS05: The "ERR" extension in "rate" data products (and other error estimates downstream) is incorrect. | Error values are estimated incorrectly by the pipeline. | Bootstrap uncertainty from science spectra. | N/A | Created issue The root cause of incorrect error estimates in the pipeline is being investigated. |
| MR-LRS06: Target acquisition images taken with FASTGRPAVG readout patterns are incorrectly calibrated. | The calibration pipeline does not compute the exposure time for these readout modes when fitting a slope to the uncalibrated data. | The verification images are calibrated correctly; if additional photometry is required, the MIRI team recommends the use these images. | N/A | Created issue The issue will be fixed in the calibration pipeline code in early 2024. |
| MR-LRS02: Extracted 1-D spectra ("x1d" files) often show salt-and-pepper noise (i.e., spikes and divots one pixel across). | Pixels flagged as "DO_NOT_USE" appear as "NaN" in the spectral images, and the signal in those pixels is missing from the summed signal at that wavelength in the extracted spectra. | Users can apply the new pixel_replace step in the developmental pipeline, which replaces the pixels flagged "DO_NOT_USE" with pixels in the same column by normalizing nearby spatial profiles in the 2-D spectral images. The best solution is to download the developmental program and just run it. The default for pixel_replace is to use 3 rows above and below the problematic row. This value can be changed by setting the n_adjacent_rows argument. | Updated Operations Pipeline The new pixel_replace step was implemented in the Operations Pipeline, installed on August 24, 2023. STScI reprocessed affected data products, which typically takes 2–4 weeks after the update. | |
| MR-LRS01: Spectra extracted from LRS slit and slitless data ("x1d" files) can have little or no signal, or even negative signal, even though the 2-D spectral images look fine. | The pipeline is extracting the spectrum from the wrong location in the 2-D spectral images because the positions it is using to determine the location of the source are inaccurate. | Users can set the location and the width of the extraction apertures themselves when running calwebb_spec2 or calwebb_spec3 manually. A notebook demonstrating the spectral extraction capabilities of the JWST calibration pipeline for MIRI LRS is available in this repository. | Updated Operations Pipeline The pipeline was updated in May 2023 to default to apertures centered on the nominal position of the level 3 spectrum in the aperture. This change is fully in operations, data have been reprocessed. |
MIRI medium resolution spectroscopy (MRS) issues
The information in this table is excerpted from MIRI MRS Known Issues
| Symptoms | Cause | Workaround | Fix Build | Mitigation Plan |
|---|---|---|---|---|
MR-MRS04: Spectra show residual regular periodic amplitude modulations. See Fringing section above. | MRS experiences significant spectral fringing, which varies with the astronomical scene and cannot be automatically corrected in its entirety for all science targets. | Run the 2-D or 1-D (preferred) residual_fringe correction steps available in the pipeline (jwst 1.11.0 onwards). This is discussed in greater detail in the Fringing section above. | N/A | Created issue None; additional non-default corrections are science case specific. Calibrations programs will explore possible future mitigations. |
MR-MRS05: Spectra extracted from small spatial regions show amplitude modulations of variable frequency (distinct from ordinary spectral fringes). See Resampling section above. | The MRS is not Nyquist sampled, and resampling the raw data to a rectified data cube introduces artifacts if extracting spectra on scales smaller than the PSF. See detailed discussion by Law et al. 2023. | Extract spectra from apertures comparable to PSF in width. | N/A | Created issue This is still under investigation; some science cases may permit scene modeling to mitigate impact. |
| MR-MRS01: World coordinate system (WCS) of data cubes is incorrect. See WCS Accuracy section above. | WCS is typically incorrect by 0.3" and in some cases more than 1" due to incorrect guide star information. | None. | N/A |
Investigate providing a notebook to update MRS WCS based on simultaneous imaging data. |
MIRI time-series observations issues
Users should assume that all pipeline issues mentioned in this article for imaging, LRS, and MRS modes also apply to TSOs with those modes. Listed here are additional issues that are TSO-specific.
The information in this table is excerpted from MIRI Time-Series Observations Known Issues
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| MR-TS02: Discontinuities in persistence behavior are seen in the SLITLESSPRISM subarray at the start of the exposure. | This is currently under investigation. | Users should consider increasing the detector settling time from 30 mins to 1 hour. This is particularly important for phase curves. | N/A | Created issue As soon as the cause of this problem is understood, the MIRI team will make the required changes to mitigate it. |
| MR-TS01: Spatially and temporally varying noise in subarray TSOs ("390 Hz noise") | This noise originates in the detector electronics. This affects subarrays SLITLESSPRISM, SUB64, and SUB128 subarrays. | The noise can be subtracted to a large extent by measuring and subtracting a background for each individual integration. Some residual noise will remain. | 10.1 | Updated issue Testing is currently underway on an algorithm that fits and removes the 390 Hz noise signal. This algorithm should be in the Operations Pipeline in early 2024. |
| MR-TS03: The photon count rate and derived flux for MRS TSO data, is lower than predicted at long wavelengths, with maximum deficit roughly a factor of 2 at 28 µm. | MRS sensitivity at long wavelengths is decreasing with time. | There is no work around at the moment.
| Updated Operations Pipeline The science calibration pipeline was modified, on December 5, 2023, to apply the time-dependent throughput correction for TSO data. STScI will reprocess affected data products with the updated Operations Pipeline; reprocessing of affected data typically takes 2–4 weeks after the update. |
NIRCam
General issues for NIRCam
The information in this table is excerpted from NIRCam Known Issues.
| Symptoms | Cause | Workaround | fix build | Mitigation Plan |
|---|---|---|---|---|
| NC-G01: Bright objects in the field are saturated even though the ETC predicted that they would not saturate "fully." | For most NIRCam readout patterns, there are many frames included in a group. Bright objects may saturate by the end of a group, but remain unsaturated in the first frame (frame0). | The calwebb_detector1 pipeline includes an option to correct an image using the unsaturated frame0 exposure, but this is turned off by default. To turn it on, rerun calwebb_detector1 with ramp_fit parameter suppress_one_group set to False. Note that frame0 is currently only used as an extra data point for saturated ramps. | N/A | Updated issue The NIRCam team will evaluate whether the frame0 correction should be turned on as the default for ramps affected by saturation; however, the investigation is still in progress due to higher priority issues. |
NIRCam time series (grism and imaging) issues
The information in this table is excerpted from NIRCam Time-Series Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NC-TS03: An excessive number of pixels are flagged as outliers in the subarray data, leading to a lower signal-to-noise ratio of the "rate" products (slope images) per integration. | Some subarrays do not have reference pixels on all sides. Without a reference pixel correction, the data becomes noisier and the jump step in calwebb_detector1 sometimes identifies too many pixels as outliers. | Rerun the jump detection step in calwebb_detector1 with an increased rejection_threshold (default is 4.0). | N/A | Updated issue The jump step algorithm and default parameters are continually being examined and optimized; improvements are expected in future builds (winter 2023 and beyond). |
NIRCam imaging issues
The information in this table is excerpted from NIRCam Imaging Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NC-I02: Some of the mosaic tiles in the stage 3 products are well aligned, but a subset of the tiles are offset. | Guiding on different stars between mosaic tiles can sometimes cause a misalignment. | In calwebb_image3, try adjusting the tweakreg fit_geometry parameter to rscale or general, which provides more flexibility in how the images are adjusted/oriented. Also try adjusting the separation and tolerance parameters to provide more stellar matches between images. Alternatively, run JHAT separately on the well-aligned and poorly-aligned data. Then, feed all data into calwebb_image3. Be sure to turn off the tweakreg step since the data have already been aligned. | N/A | Created issue Mitigation is not yet scheduled due to higher priority issues. |
| NC-I03: There are large scattered light features on the images of some NIRCam detectors. | There are several NIRCam scattered light artifacts. The most common are claws and wisps, which are caused by light entering directly through the aft optic system (AOS) mask, located in front of the JWST tertiary mirror, without first bouncing off the primary and secondary mirror. | For claws: Avoid position angles that place bright stars in the susceptibility zone. For wisps: Wisps can be subtracted using templates, available to download here: NIRCam Claws and Wisps | N/A | Updated issue For claws: The mitigation plan for removing claws from data is not yet scheduled. However, the NIRCam team is screening all programs to minimize the risk of claws before programs are observed. For wisps: New wisp templates using ~2 years of extra data compared to the previous versions are now available. |
| NC-I04: An excessive number of pixels are flagged as outliers in some subarray data. | Some imaging subarrays do not have reference pixels on all sides, especially the extended source subarrays in the long wavelength channel. Without a reference pixel correction, the data become noisier and the jump step in calwebb_detector1 sometimes identifies too many pixels as outliers. | Rerun the jump detection step in calwebb_detector1 with an increased rejection_threshold (default is 4.0). | N/A | Updated issue The jump step algorithm and default parameters are continually being examined and optimized; improvements are expected in future builds. |
| NC-I06: Point source aperture photometry results may show a filter-dependent discrepancy compared to PSF photometry results when using NIRCam aperture correction reference files in CRDS. | Aperture correction reference files (APCORR) used in the source_catalog step have not yet been updated using flight data. | Use PSF photometry, if possible. An example is available in the JWST Data Analysis Tool Notebooks: NIRCam PSF Photometry Example. The discrepancy is worse for smaller radii and shorter wavelength filters. For larger apertures and longer wavelength filters, results using aperture photometry and PSF photometry should match to within a few percent. | N/A | Created issue The NIRCam team is working on updating the aperture correction reference files (late-summer/fall 2024). |
| NC-I07: Some tiles in a mosaic show very large (arcmin) errors in their WCS. | This is often due to missing engineering data in the original pipeline run. | Call the set_telescope_pointing.py script on the uncal or rate files. Then re-run the remaining pipeline stages. | N/A | Updated issue This is a very rare failure mode. The workaround is sufficient to correct cases where the failure occurs. |
| NC-I08: Inconsistent background levels from integration to integration with subarray data | Few or no reference pixels associated with the subarray being used can lead to an inconsistent background. | Use the skymatch step of the stage 3 calibration pipeline to make background levels consistent. | N/A | |
NIRCam coronagraphic imaging issues
The information in this table is excerpted from NIRCam Coronagraphy Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NC-CI01: Dark correction leads to worse outcome than not applying any. | On-sky darks are not high enough in SNR and lead to residuals and erroneous jumps in the "rateints" images. | Skip dark correction during calwebb_detector1. | N/A | Created issue Mitigation is not yet scheduled because the data are satisfactory without dark subtraction. |
| NC-CI02: An excessive number of pixels are flagged as outliers in some subarray data. | The coronagraph subarrays do not have reference pixels on all sides. Without a reference pixel correction, the data become noisier and the jump step in calwebb_detector1 sometimes identifies too many pixels as outliers. | The spaceKLIP open-source package (provided by the community) implements a pseudo-reference pixel correction using a few not-illuminated pixels around (at the edge of) the subarray and forces them to be reference pixels. This way, global DC offsets frame to frame are taken care of. Users can also increase the rejection_threshold in the calwebb_detector1 jump step to reduce the number of outlier flags. | N/A | Updated issue Implement something similar to spaceKLIP for calwebb_coron3. Timeline TBD. |
NIRCam wide field slitless spectroscopy (WFSS) issues
The information in this table is excerpted from NIRCam WFSS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NC-WFSS01: Adjacent sources are included in extracted spectra. | This occurs in crowded fields with overlapping spectral traces. | No workaround is available yet. The calwebb_spec2 pipeline includes an option to provide a first-order correction for source contamination with the wfss_contam step, but some discrepancies and issues are still being investigated. | N/A | Updated issue Implement the wfss_contam step in calwebb_spec2. Work on this is ongoing and is planned for completion in 2024. |
| NC-WFSS02: The fluxes of some spectra in the field appear to be too low or even negative. | This is caused by over-subtraction of the background. The background subtraction step in calwebb_spec2 assumes the background is constant and does not vary in time, which can be a limiting factor for low signal-to-noise observations. | Users can manually perform a local background subtraction on extracted 2-D spectra before the 1-D extraction in calwebb_spec2. | N/A | Created issue A mitigation plan is under development. |
| NC-WFSS04: Calibrated flux values in WFSS or grism time series products are too large or too small by a factor of ~10. | Versions of the pipeline prior to 1.14.0 require the use of older PHOTOM reference files (jwst_nircam_photom_0129.fits and jwst_nircam_photom_0130.fits). However, when re-running the pipeline, CRDS may download newer reference files designed for pipeline version 1.14.0 or later. Version 1.14.0 of the calibration pipeline requires the use of the new photom reference files (jwst_nircam_photom_0162.fits and jwst_nircam_photom_0163.fits. | STScI recommends that users upgrade to the latest version of the pipeline, and allow CRDS to download and use the most recent PHOTOM reference files. | N/A | N/A. |
NIRISS
NIRISS aperture masking interferometry (AMI) issues
The information in this table is excerpted from NIRISS AMI Known Issues.
| Symptoms | Cause | Workaround | Fix Build | Mitigation Plan |
|---|---|---|---|---|
NR-AMI01: Interferometric properties returned by the calwebb_ami3 stage of the pipeline require further evaluation. Data are served in FITS files rather than the interferometric convention to use OIFITS formatted files. | The Science Calibration Pipeline is using an older version of the image plane reconstruction software. | The JWST Aperture Masking Interferometry Pipeline Caveats article provides descriptions of how to analyze data products off-line after the calwebb_image2 stage of the Science Calibration Pipeline, with links to analysis tools. | 10.2 | Updated issue The calwebb_ami3 stage of the Science Calibration Pipeline was updated to use state-of-the-art code and to serve OIFITS formatted files. STScI is reprocessing affected data products with an updated Operations pipeline, installed on XX 2024. Reprocessing of affected data typically takes 2–4 weeks |
NR-AMI02: When the peak pixel of a PSF in AMI mode reaches beyond ~25,000 ADU in an integration, it starts "spilling" charge to its neighboring pixels, thus causing an effective "widening" of the PSF. | This is due to the so-called "brighter-fatter effect" (BFE) that affects near-IR H2RG detectors. | No generic workaround is available until an Operations Pipeline build planned for installation in November 2023. For advice on specific datasets for which the peak intensity reaches beyond ~25,000 ADU per integration, a help desk ticket can be submitted. The NIRISS AMI Recommended Strategies article in JDox describes how to avoid this problem at the proposal planning stage. | Updated issue Apply a new Science Calibration Pipeline step called charge_migration within the calwebb_detector1 stage (Goudfrooij et al. 2024). STScI is reprocessing affected data products with an updated Operations pipeline, installed on December 5, 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. |
NIRISS single object slitless spectroscopy (SOSS) issues
The information in this table is excerpted from NIRISS SOSS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
NR-SOSS05: Observations show a background that abruptly increases in flux at around column ~700. | This is due to the background (typically dominated by zodiacal light background at NIRISS/SOSS wavelengths) being dispersed by the GR700XD grism into the detector (see Albert et al., 2023; Section 8.5). | Scale the background taken during commissioning for SUBSTRIP256 and for SUBSTRIP96 to observations using non-illuminated regions of the detector. | N/A | Created issue Calibration program 4479 is studying background templates at different positions in the sky, as well as the variation of the background due to the pupil wheel position (PWCPOS) varying from visit to visit. Implementation on the JWST pipeline is still under development. |
NR-SOSS06: There is evident banding in the cross-dispersion direction, which varies from group-to-group and integration-to-integration. | This is due to residual, uncorrected 1/f noise left over from the reference pixel correction step. | There are several community tools available designed to remove 1/f noise. | N/A | Created issue A mitigation plan is under development. |
NR-SOSS07: Large number of outliers identified in the jump detection step, which cause lower SNR in the "_rateints" (count rate per integration) products. | The default parameters of the jump step in the JWST pipeline are too aggressive and flag too many pixels as cosmic rays. | Re-run the jump detection step in calwebb_detector1 with a rejection_threshold value > 10 (default is 4.0). | N/A | Created issue A mitigation plan is under development. |
NR-SOSS08: The pipeline is unable to extract a SOSS spectrum with the GR700XD/F277W combination. | The pipeline currently does not support this grism/filter combination. | There is currently not a workaround. | N/A | Updated issue As of pipeline build 11.0, the pipeline no longer throws an error when processing SOSS data using the GR700XD/F277W combination and instead exits gracefully at the end of the level 2a stage. Updates to the pipeline code to support the GR700XD/F277W combination are under development. |
NR-SOSS01: Too many pixels are flagged as "DO_NOT_USE" which are then set to "NaN" values in the science array. Some of these pixels are fine for SOSS data analysis. | Several detector-level reference files (mask, linearity, saturation, super bias) need to be updated to not cause certain pixels to be flagged as "DO_NOT_USE". | Use the attached Jupyter notebook to turn off setting pixels to "DO_NOT_USE". (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated Operations Pipeline New reference files were delivered to CRDS which no longer erroneously flag good pixels as "DO_NOT_USE". The new linearity and saturation reference files are available in CRDS contexts "jwst_1152.pmap" and later. The new subarray superbias reference files are available in CRDS contexts "jwst_1170.pmap" and later. Affected data products will be reprocessed; reprocessing of affected data typically takes 2–4 weeks. | |
NR-SOSS02: The shape of the spectral trace varies from visit to visit for all the NIRISS/SOSS orders. Distortion is higher at longer wavelengths. This might lead to lower SNR in the extracted spectra. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the shape of the spectral trace. | Use the PASTASOSS package to predict spectral traces given the PWCPOS, or use the package as a starting point to perform spectral tracing and thus extraction. | Updated issue Fixed in JWST pipeline build 11.1.1. | |
NR-SOSS03: The wavelength solution varies from visit to visit, appearing to be shifted by a handful of pixels. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position; this causes a distortion in the wavelength solution. | Use the PASTASOSS package to predict the wavelength solution for the visit. | Updated issue Fixed in JWST pipeline build 11.1. | |
NR-SOSS04: The shape of the spectral trace and wavelength solution sometimes varies from exposure to exposure on a given visit, including between subsequent CLEAR and F277W exposures. | The actual pupil wheel position (PWCPOS) that sets the GR700XD grism in the optical path on each visit does not land exactly at the commanded position. When the position lands above a certain threshold from the commanded position, the onboard mechanism tries to re-adjust the position between exposures until it meets the threshold. | If trying to match CLEAR and F227W exposures, perform a translation and/or rotation of the F227W exposure to match the CLEAR exposure. Use stellar lines to match the wavelength solution on the F277W exposure. | N/A | Updated Operations Pipeline The NIRISS team has implemented the addition of a "Short First Exposure" consisting of a single group, single integration at the beginning of every exposure that requires an F277W exposure. This will give the on-board mechanism the opportunity to re-adjust the pupil wheel position if needed before the science exposures. This was implemented in the NIRISS SOSS template in APT 2023.5 (release date: August 24, 2023). |
NIRISS wide field slitless spectroscopy (WFSS) issues
The information in this table is excerpted from NIRISS WFSS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
NR-WFSS01: Spectral traces are offset from the sources in the cross-dispersion direction. | The current "specwcs" reference file (as of CRDS version jwst_1176.pmap) incorporates the field dependence in the wavelength dispersion solution. However, the shape of the spectral traces also vary as a function of detector position. As of CRDS version jwst_1228.pmap, the field dependence in the spectral trace shape is calibrated for the GR150R/F200W and GR150C/F200W combinations. | None. | Updated issue In-depth investigation is underway to calibrate the change in shape of the spectral traces as a function of detector position for the remaining filter/grism pairs. | |
NR-WFSS02: For direct imaging: when the peak pixel of a star (or other compact object) reaches beyond ~25,000 ADU in an integration (after bias and dark subtraction and linearity correction), it starts "spilling" charge to its neighboring pixels. This causes an effective "widening" of the PSF or charge distribution, as well as flux loss for those objects in the combined, resampled products of the calwebb_image3 stage of the Science Calibration Pipeline. This effect is strongest for the most undersampled modes (i.e., filters with pivot wavelength <= 2 μm). | This is due to the so-called "brighter-fatter effect" (BFE) that affects near-IR H2RG detectors, in combination with the current way "jumps" are detected and dealt with in the calwebb_detector1 pipeline stage, and how the latter affects image combination in the calwebb_image3 pipeline stage. | No efficient workaround is available at present. | Updated Operations Pipeline Apply a new Science Calibration Pipeline step called charge_migration within the calwebb_detector1 stage (Goudfrooij et al. 2024). STScI is reprocessing affected data products with an updated Operations pipeline, installed on December 5, 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
NR-WFSS03: There is an offset in the wavelength zeropoint that is most pronounced in the F200W filter. | The filteroffset reference files, which accounts for filter-to-filter offsets in the astrometric solution, does not include WFSS exposures. | Updated Operations Pipeline Updated filter offset files that include WFSS exposures were delivered to CRDS and are available as of CRDS version jwst_1176.pmap and later. | ||
NR-WFSS04: WCS keywords in the header of pure parallel WFSS observations (both direct images and grism exposures) taken prior to June 12, 2024, are "off" by of order 0.1 arcsec no matter which SI is prime, and the offset is generally different for different dither positions. | In JWST data, the header keywords that are used to determine precise WCS information use the guide star coordinates. In pure parallel data taken prior to June 12, 2024, these coordinates were unknown prior to execution. This is why for those pure parallel data, the WCS were determined by using a "coarse" algorithm, leading to the accuracy issue. | For the data affected by this issue, a Python script has been made available in this GitHub repository to correct the relevant WCS header keywords in the products of the “calwebb_detector1” pipeline, i.e., the “_rate.fits” and/or “_rateints.fits” files. After that script has been run on the input file(s), one can then (re-)run Stage 2 of the JWST pipeline (“calwebb_image2” and/or “calwebb_spec2”) on those files, which will then have correct WCS header information. | 10.2 (for data taken after June 12, 2024) | Updated Operations Pipeline This issue was fixed in JWST operations as part of build 10.2 of the JWST Science Calibration Pipeline (June 12, 2024). Data taken after that date do not have this issue anymore. |
NIRISS Imaging issues
The information in this table is excerpted from NIRISS Imaging Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
NR-I01: When the peak pixel of a star (or other compact object) reaches beyond ~25,000 ADU in an integration (after bias and dark subtraction and linearity correction), it starts "spilling" charge to its neighboring pixels. This causes an effective "widening" of the PSF or charge distribution, as well as flux loss for those objects in the combined, resampled products of the calwebb_image3 stage of the Science Calibration Pipeline. This effect is strongest for the most undersampled modes (i.e., filters with pivot wavelength <= 2 μm). | This is due to the so-called "brighter-fatter effect" (BFE) that affects near-IR H2RG detectors, in combination with the current way "jumps" are detected and dealt with in the calwebb_detector1 pipeline stage, and how the latter affects image combination in the calwebb_image3 pipeline stage. | No efficient workaround is available at present. | Updated Operations Pipeline Apply a new Science Calibration Pipeline step called charge_migration within the calwebb_detector1 stage (Goudfrooij et al. 2024). STScI is reprocessing affected data products with an updated Operations pipeline, installed on December 5, 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
NR-I02: WCS keywords in the header of pure parallel imaging observations taken prior to June 12, 2024, are "off" by of order 0.1 arcsec no matter which SI is prime, and the offset is generally different for different dither positions. | In JWST data, the header keywords that are used to determine precise WCS information use the guide star coordinates. In pure parallel data taken prior to June 12, 2024, these coordinates were unknown prior to execution. This is why for those pure parallel data, the WCS were determined by using a "coarse" algorithm, leading to the accuracy issue. | For the data affected by this issue, a Python script has been made available in this GitHub repository to correct the relevant WCS header keywords in the products of the calwebb_detector1 pipeline, i.e., the “_rate.fits” and/or “_rateints.fits” files. After that script has been run on the input file(s), one can then (re-)run stage 2 of the JWST pipeline (calwebb_image2 and/or calwebb_spec2) on those files, which will then have correct WCS header information. | 10.2 (for data taken after June 12, 2024) | Updated Operations Pipeline This issue was fixed in JWST operations as part of build 10.2 of the JWST Science Calibration Pipeline (June 12, 2024). Data taken after that date do not have this issue anymore. |
NIRSpec
General issues for NIRSpec
The information in this table is excerpted from NIRSpec Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-G03: Extracted 1-D spectrum ("x1d" product) has spurious positive and negative features, typically one pixel wide. | Pixels marked as "DO_NOT_USE" in reference data are not included in the 1-D extracted spectrum. | Workaround: Use the new Science Calibration Pipeline software (jwst 1.11.0 onwards) to estimate the value for each "DO_NOT_USE" pixel from neighboring 2-D profiles before extracting the 1-D spectrum by running the pixel_replace step. This is disabled by default. Testing has shown improvements in many cases, but degradation in data quality has also been seen in certain circumstances; users should exercise caution if they choose to run the step. | N/A | Updated issue The team is evaluating options for further improvements. |
| NS-G06: A pattern of alternating brighter/darker pixel rows over part or all of a detector image. | An effect sometimes referred to as alternating column noise (ACN; note that pixel rows in the science data actually represent columns in the detector frame of reference, as the data are rotated to put the dispersion direction horizontally). This is due to the two amplifiers (one for odd and one for even columns) in an output sometimes having slightly different offsets because of drift or a cosmic ray event. | For traditional readout modes calwebb_detector1 refpix.odd_even_columns is on by default. If alternating column noise is seen in data taken in these modes, it may be improved by turning off refpix.odd_even_columns. For IRS2 readout modes refpix.odd_even_columns is disabled by default. As of jwst 1.14.0 and later refpix.odd_even_columns can be used for IRS2 modes. Testing has shown improvements in some cases, but degradation in data quality has also been seen in certain circumstances; users may try reprocessing with this setting on, but should exercise caution if they choose to do so. | N/A | Updated issue The team is investigating additional improvements to the treatment of alternating column noise in IRS2 modes. |
NS-G09: Uncertainties in the combined background images produced in the background step are not propagated into science data. | The background step does not propagate uncertainties from the background images to the 2-D science arrays | Uncertainties from the the combined background images can be recovered by saving the combined background using background.save_combined_background = True and combining the uncertainties from this image with the science data uncertainties from the background step. | N/A | Created issue Work on a fix is pending an investigation into the necessary implementation. |
| NS-G01: A few pixel rows exhibit anomalous count rates in "rate" images for IRS2 data. | Some reference pixels are "bad" (e.g., excessive noise or "telegraph" behavior) and are flagged as such. However, those flags are not considered in the refpix step of the calwebb_detector1 pipeline, resulting in anomalous corrections for the associated science pixels. | None. | Updated Operations Pipeline A flag check has been implemented, along with an improved algorithm to filter out noisy reference pixels. STScI will reprocess affected data products with an updated Operations Pipeline, planned for installation on . Reprocessing of affected data typically takes 2–4 weeks after the update. | |
| NS-G02: No flux or negative flux in FS or MOS extracted 1-D spectrum ("x1d" product), despite obvious flux in 2-D spectrum ("s2d" product). | Extraction aperture is offset along the spatial axis, for example, due to an error in target coordinates or the world coordinate system. For targets with large proper motion, see | For high proper motion targets, if the processing date ( Rerun the extract_1d step with a modified "extract1d" reference file containing the desired extraction aperture location (using either ystart and ystop or src_coef) and the parameter use_source_posn set to False. See this workaround notebook for an example of how to do this using FS data. | Updated Operations Pipeline Updates to spacecraft pointing keywords, which should improve the spectral trace WCS, have been available in the Operations Pipeline since on 24 August 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. An alternate method of extraction aperture centering for MOS and FS modes was implemented, based on the planned slit position of each source, therefore avoiding sky coordinates altogether. This new algorithm was installed in the Operations Pipeline on December 5, 2023. | |
| NS-G04: Unexpected flux levels in spectra of sources not centered in a slit. | A bug was found in the code used to interpolate the pathloss correction from the "pathloss" reference data. Affects FS and MOS point source data. | None. | Updated Operations Pipeline A bug fix was implemented and made available in the Operations Pipeline installed on December 5, 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
| NS-G05: Wavelengths of spectral features do not match expectation in spectra of sources not centered in a slit. | The calwebb_spec2 pipeline performs a wavelength correction for offset point sources based on their expected position within the slit. However, the correction values are not being propagated to the resample_spec step, so the "s2d" products always have the wavelength scale appropriate for centered sources only. Affects FS and MOS point source data. | Reprocess spectra using the new version of the calibration pipeline software (jwst 1.15.0 or later). The wavelength corrections for point sources are now applied correctly and propagated to the resample_spec step. There are no such corrections applied for extended sources, whose wavelength corrections depend on the source geometry. | Updated Operations Pipeline A fix for this issue has been implemented in the jwst calibration software (jwst 1.15.0 and later) and was made available in the Operations Pipeline installed on August 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
| NS-G07: In some data from the NRS2 detector taken with NRSIRS2 readout patterns, users may notice an entire row of pixels with the incorrect DQ flag value, and an excess of good pixels in contiguous areas incorrectly flagged as "hot". | The mask reference file for the NRS2 detector and NRSIRS2/NRSIRS2RAPID readout patterns, covering all data taken after , was incorrectly generated. The snowball correction was not applied to the dark data from which the hot pixel population is determined, leading to many pixels affected by snowball events being inadvertently flagged as "hot". In addition, a file conversion error led to one row of pixels to be flagged incorrectly as reference pixels. Note that there is no effect on the science data, other than that a small excess of pixels will be rejected during stage 3 combination. | Re-process data using the updated calibration reference files in CRDS context jwst_1234.pmap or later. | Updated Operations Pipeline A corrected reference file was delivered to CRDS and was available starting. These files are used with pipeline build 10.2 released mid-June. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
| NS-G08: Data with associated background or imprint observations that is reprocessed locally with 1/f cleaning using the nsclean step does not remove 1/f noise. | The nsclean step in calwebb_spec2 currently only runs on the science members in an association. Background and imprint subtraction reintroduces 1/f noise, because these members in the association have not been cleaned of 1/f noise through the nsclean step. | Run calwebb_spec2 with nsclean on all science and background members, save the results of the nsclean step, and rerun calwebb_spec2 on association files that use the 1/f cleaned rate files as inputs. | Updated Operations Pipeline A fix for this issue has been implemented in the jwst calibration software (jwst 1.16.0 and later). Associations reprocessed with nsclean will now run the nsclean step on all science, background, and imprint members in an association. |
NIRSpec fixed slit (FS) spectroscopy issues
The information in this table is excerpted from NIRSpec FS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-FS05: Many significant outliers appear in the 2-D and 1-D extracted spectra. | The outlier_detection step generally has a hard time finding many outliers. Step parameters need to be tuned to the noise characteristics of each data set, although in many cases outliers are still missed. | For outlier improvement, rerun the outlier_detection step in calwebb_spec3 with different values of the snr parameter. | N/A | Updated issue Updated algorithms are under investigation, possibly for inclusion in the Operations Pipeline update planned for the end of February 2025. |
NS-FS07: Lower than expected S/N and/or larger than expected discrepancies between dither positions. | Use of the default “inverse variance weighting” using read noise variance in the resample_spec step (resample_spec.weight_type = ivm) does not appear to be appropriate for high signal-to-noise data | When running calwebb_spec2, set resample_spec.weight_type = exptime, and when running calwebb_spec3, set spec3.weight_type = exptime. | Updated issue The default resampling weight has been set to exptime as of JWST CRDS context jwst_1252.pmap (and later), which is included in the Operations Pipeline update on August, 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. In the longer term consider extraction and image combination algorithms that don’t require resampling the 2-D data, or extraction options with improved centroiding. | |
NS-FS08: Lower than expected flux and/or flux inconsistencies between closely spaced dither positions that may result in issues with outlier rejection. In extreme cases this may result in an ~20% error in the absolute flux calibration, although more commonly, the effect is at the ~5% level. | When closely spaced nod positions are used as background members for each other, some source flux is subtracted from the extraction region along with the background, but the fraction of source flux subtracted varies with the dither separation of each background member. Currently the pipeline does not account for this in the flux calibration. | For sufficiently bright sources, consider omitting background members in the level 2 association, and/or find separate background exposures from other observations. Otherwise, when using 5 primary dither points, especially in the S1600A1, edit level 2 association to remove background members separated by less than about 0.375” from the science member in that association. Note that the pipeline already knows not to use sub-pixel spatial or dispersion direction dithers as background members for other exposures at the same primary dither position. | N/A | Updated issue Modify default pipeline rules to enforce a minimum separation for background members used in calwebb_spec2. In the longer run, investigate adding a step to calwebb_spec2 to estimate the flux loss due to subtraction of other dithers and correct the flux in the "cal" files to account for this. Once fixes are in place to minimize these effects, reconsider all F-flat reference files for FS, (and possibly also MOS), to allow improved absolute and relative flux accuracy. Updated algorithms are under investigation, possibly for inclusion in the Operations Pipeline update planned for December 2024. |
NS-FS09: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture. | The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture. | Use only the spectra extracted from the upper dither positions in the affected wavelength range. | N/A | Created issue Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE. |
NS-FS10: Discrepancies in extracted 1D fluxes at the 3-5% level when changing the assign_wcs step slit width. | There is a bug in the NIRSpec spectral resampling that causes the output pixel area to differ from the input pixel area depending on the assign_wcs slit width. | Reprocess spectra using the new version of the calibration pipeline software (jwst 1.16.0or later and using JWST CRDS context jwst_1298.pmap or later). The spectral resampling now conserves flux regardless of slitlet length. | N/A | Updated issue The spectral resampling has been improved in jwst 1.16.0 (and later versions) and is set to be included in the Operations Pipeline update in January 2025. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. Any users reprocessing data with jwst 1.16.0 (and later versions) should use JWST CRDS context jwst_1298.pmap (or later), which includes updated flux calibrations for the new spectral resampling code. |
NS-FS11: Level 3 spectra that fall on NRS1 and NRS2 (H-grating data) show unexpected outliers towards the blue end of NRS2 data. | Because the read noise variances differ between NRS1 and NRS2, when using the default read noise variance weighting, outlier_detection.weight_type = ivm, and outlier_detection.maskpt = 0.7, the outlier_detection step may mask parts of the observed spectra on NRS2, producing unexpectedly noisy spectra in the masked regions. | When running calwebb_spec3 either turn off the outlier_detection step or set outlier_detection.weight_type = exptime. | N/A | Created issue A fix for this has been identified and is expected to be included in the Operations Pipeline update at the end of February 2025. |
| NS-FS03: Spectra obtained with the SLIT = S200A1 and S200A2 option in APT are not combined to create a single spectrum with continuous wavelength coverage. | Association logic does not realize that SOURCEID = 1 observed with S200A1 is the same target as SOURCEID = 2 observed with S200A2. | Update | updated Operations Pipeline Enhanced the association logic and changed the | |
| NS-FS06: Level 3 extracted spectra have errors that are all "NaN." | The flat field reference file uncertainties are currently zero. The resultant flat error component calculated from these is "NaN", which propagates to the combined error as "NaN". | Recalculate the combined error using only the read noise and photon noise components; instructions will soon be available. See this worked example for more on how to do this. (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated issue, new due date for reference file Reprocess data with an enhanced calibration reference file (flat) in CRDS. An update is planned for early 2024. Reprocessing of old data typically takes 2–4 weeks after the update. | |
| NS-FS04: FS spectra obtained with the G140M grating exhibit flux discrepancies of up to 20%. | No spectrophotometric observations for this grating were obtained during commissioning, so the F-flat reference file (flux calibration) for this specific case is the model-based pre-flight version. | None. | Issue resolved Spectrophotometric observations with this configuration were obtained in August, and updated F-flat reference file were delivered on August 24, 2023. | |
| NS-FS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. | Some subarrays have no reference pixels, which means bias drifts are not corrected. | Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps. | Updated Operations Pipeline The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. STScI reprocessed the affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-FS02: Background spectra taken from one fixed slit does not match the point source spectrum extracted from a different slit. | The pathloss step as applied to FS data had a bug that led to the correction being applied twice. This affects all FS exposures; the impact on point sources is minor (a few percent at most), but higher for extended sources. | None. | Updated Operations Pipeline The pathloss correction was applied only once. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) |
NIRSpec integral field unit (IFU) spectroscopy issues
The information in this table is excerpted from NIRSpec IFU Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-IFU05: Spectra extracted from single spaxels on/near point sources show a sinusoidal modulation. | NIRSpec is undersampled, and distortion causes spectral traces (particularly for the gratings) to be curved on the detector. Resampling the raw data to a rectified data cube introduces artifacts if extracting spectra on scales smaller than the PSF. See detailed discussion by Law et al. 2023. | Extract spectra from larger apertures comparable in width to the PSF. Combining dithers/nods also reduces, but does not eliminate, the effect. | N/A | Updated issue None at this time. |
| NS-IFU06: The centroid of point sources appears to drift by up to 0.5 pixels as a function of wavelength. | Issue related to a combination of filter transmission effects and the calibration of the IFU instrument model. The drift is typically of order 20 milliarcsec over the wavelength range of a given disperser. | None at present. | N/A | Updated issue None at this time. Any action regarding pipeline or post-processing mitigation await an investigation on the root cause, which is underway via Spatial IFU Distortion calibration program 6641. |
NS-IFU07: The spectral centroid of emission lines shows unexpected structure across IFU slices. | Issue related to how the NIRSpec IFU instrument model is anchored to the RA, Dec, wavelength mapping on the sky. | None at present. | N/A | Created issue None at this time. Any action regarding pipeline or post-processing mitigation await an investigation on the root cause, which is underway via Spectra IFU Distortion calibration program 6640. |
| NS-IFU01: There is a hole at the peak of the PSF or an otherwise distorted PSF in IFU "s3d" cube. | This is due to overly aggressive outlier detection. | Option 1: Turn off outlier rejection in the cube_build step of calwebb_spec3. However, this may allow other outliers to remain in the cube. Option 2: Reprocess data using the Science Calibration Pipeline, jwst 1.11.3 and onward. This version will be installed in the Operations Pipeline with build 9.3), planned for installation on . | Updated Operations Pipeline The outlier detection algorithm was updated. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-IFU02: Flux is not conserved in 1-D-extracted spectra of point sources when using a different output spaxel sampling. | The cube_build algorithm was designed to conserve flux assuming input units of surface brightness. However, the NIRSpec point source calibration produces units of flux density, which is not compatible. | If using data processed with the current Operations Pipeline build, do the following:
Otherwise, install the release candidate for the coming Operations Pipeline (jwst 1.11.3 and onward) and re-run calwebb_spec2. | Updated Operations Pipeline Point source calibration for IFU data was changed to surface brightness units, and 1-D spectra converted back to flux density. STScI reprocessed affected data products with an updated Operations Pipeline that was installed on August 24, 2023. (Reprocessing of affected data typically takes 2–4 weeks after the update.) | |
| NS-IFU03: Negative and/or surplus flux in the extracted 1-D spectra is seen, typically with an irregular wavelength-dependent undulation. | Correlated noise from low-level detector thermal instabilities, seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector. While the IRS2 readout modes reduces this effect, it is not completely eliminated. | Run the NSClean script developed by B. Rauscher on count rate images, using an appropriate mask. (Rauscher, B. 2023, arXiv:2306.03250) A notebook demonstrating the use of the NSClean algorithm is now available. | Updated workaround notebook link NSClean was implemented in the pipeline (v1.13.4 onwards) as a non-default option. Details on how to invoke NSClean within the science calibration pipeline are provided in the workaround notebook. | |
| NS-IFU04: There is missing flux in the "x1d" spectrum of point sources. | Astrometric or pointing uncertainty may cause the default extraction aperture to miss the intended target or be off-center. | Option 1: Rerun the extract_1d pipeline step in calwebb_spec3 at specified coordinates. Option 2: Install the latest release of the jwst package and then run the Science Calibration Pipeline on the affected dataset. Starting in jwst 1.11.0, the extract_1d step supports setting ifu_autocen = True. | Updated issue An IFU astrometric solution was updated in July 2023, but small pointing offsets may remain. Reprocessing of affected data typically takes 2–4 weeks after the update. |
NIRSpec multi-object spectroscopy (MOS) issues
The information in this table is excerpted from NIRSpec MOS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-MOS01: Significant outliers appear in the 2-D and 1-D extracted spectra. | The outlier_detection step generally has a hard time finding many outliers. Step parameters need to be tuned to the noise characteristics of each data set, although in many cases outliers are still missed. | For outlier improvement, rerun the outlier_detection step in calwebb_spec3 with different values of the snr parameter. | N/A | Updated issue Updated algorithms are under investigation, possibly for inclusion in the Science Calibration Pipeline in the end of February 2025. |
| NS-MOS04: Negative and/or surplus flux seen in extracted 1-D spectra, typically with an irregular wavelength-dependent undulation. | Correlated noise from low level detector thermal instabilities, seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector. While the "IRS2" readout modes reduce this effect, it is not completely eliminated. | It may be possible to improve the noise levels using the NSClean script developed by B. Rauscher on count rate images, using an appropriate mask. A notebook demonstrating the use of the NSClean algorithm is now available. Additional 1/f cleaning options in the pipeline have been included in jwst 1.16.0 (and later versions), which can now be run on group data in calwebb_detector1 using the clean_flicker_noise step. | N/A | Updated issue NSClean was implemented in the pipeline and is available as a non-default option (jwst 1.13.0 and later versions). Details on how to invoke NSClean within the science calibration pipeline are provided in the workaround notebook. Investigations are underway to explore whether 1/f cleaning can be turned on by default in the pipeline. |
| NS-MOS05: Unexpected variations are seen in continuum or line fluxes as a function of field position. | The flux calibration for MOS is currently based on spectrophotometric observations at a single point in the MOS field of view. The flat field calibration accounts for spatial variations from the MSA aperture plane, through the grating wheel, and at the detector, but not in the OTE or filter wheel. Large variations (>~ 5%) are not expected, but this needs to be confirmed. | None. | N/A | Updated issue Spectrophotometric observations at multiple field points have been obtained and are being analyzed. Investigations are under way to determine whether updates to the flux calibration are needed. |
NS-MOS07: Lower than expected S/N and/or larger than expected discrepancies between dither positions. | Use of the default “inverse variance weighting” using read noise variance in the resample_spec step (resample_spec.weight_type = ivm) does not appear to be appropriate for high signal-to-noise data | When running calwebb_spec2, set resample_spec.weight_type = exptime, and when running calwebb_spec3, set spec3.weight_type = exptime. | Updated issue The default resampling weight has been set to exptime as of JWST CRDS context jwst_1252.pmap (and later), which is included in the Operations Pipeline update on August, 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. In the longer term consider extraction and image combination algorithms that don’t require resampling the 2-D data, or extraction options with improved centroiding. | |
NS-MOS09: Discrepancies in extracted 1D fluxes when extracting from MOS slitlets with different numbers of shutters at the 3-5% level. | There is a bug in the NIRSpec spectral resampling that causes the output pixel area to differ from the input pixel area depending on the slitlet length. | Reprocess spectra using the new version of the calibration pipeline software (jwst 1.16.0 or later and using JWST CRDS context jwst_1298.pmap or later). The spectral resampling now conserves flux regardless of slitlet length. | N/A | Updated issue The spectral resampling has been improved in jwst 1.16.0 (and later versions) and is set to be included in the Operations Pipeline update in January 2025. Reprocessing of affected data typically takes 2–4 weeks after the update. This improves some, but not all, discrepancies between dither positions. Any users reprocessing data with jwst 1.16.0 (and later versions) should use JWST CRDS context jwst_1298.pmap (or later), which includes updated flux calibrations for the new spectral resampling code. |
NS-MOS10: Uncertainties in the background spectrum are not accounted for when using the master_background_mos step. | The master_background_mos step does not propagate uncertainties in the background spectrum to the 2-D science arrays | It may be possible to estimate the uncertainties introduced by the background subtraction background noise by saving the 1-D background spectrum and using its uncertainties. This spectrum can be saved by setting master_background_mos.save_background = True, and will save the "x1d" spectrum into a file with the suffix "masterbg1d". | N/A | Created issue Work on a fix is pending an investigation into the necessary implementation.
|
NS-MOS11: Level 3 spectra that fall on NRS1 and NRS2 show unexpected outliers in NRS2 data. | Because the read noise variances differ between NRS1 and NRS2, when using the default read noise variance weighting, outlier_detection.weight_type = ivm, and outlier_detection.maskpt = 0.7, the outlier_detection step may mask parts of the observed spectra on NRS2, producing unexpectedly noisy spectra in the masked regions. | When running calwebb_spec3 either turn off the outlier_detection step or set outlier_detection.weight_type = exptime. | N/A | Updated issue A fix for this has been identified and is expected to be included in the Operations Pipeline update at the end of February 2025. |
| NS-MOS02: Level 3 extracted spectra have errors that are all "NaN". | The flat field reference file uncertainties are currently zero. The resultant flat error component calculated from these is "NaN", which propagates to the combined error as "NaN". | Recalculate the combined error using only the read noise and photon noise components. See this worked example for more on how to do this. (NB: this notebook has now been deprecated as the fix is in the Build 10.0 pipeline). | Updated issue New reference files were delivered to Operations Pipeline; reprocessing of affected data typically takes 2–4 weeks after the update. | |
| NS-MOS03: Spectra extracted from slitlets consisting of more than 3 shutters exhibit unexpected wavelength-dependent flux discrepancies. | A bug in the pathloss step prevents the application of the correction in longer slitlets, for either point or extended sources. | Reprocess data using the calibration pipeline software jwst 1.13.0 (or later versions). | Updated Operations Pipeline An update to the science calibration pipeline code to enable application of the current set of reference files to any slitlet length was installed in the Operations Pipeline released February 27, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
NS-MOS06: Discontinuities in the extracted level 2 spectra and "s2d" images at the upper and/or lower nod positions. These may be as large as ~40%. The effects in level 3 products appear to be much smaller and/or absent. | Errors in rectification when producing the"s2d" image is due to missing WCS information at the edges of CAL image extensions. | Edit the MSA metafile to pretend that an extra shutter at the top and bottom of the slitlet was open. | Updated Operations Pipeline A fix has been implemented and was installed in the Operations Pipeline released February 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. | |
NS-MOS08: Discrepancies in the wavelengths of known spectral features in 1D spectra from MOS observations of point sources that are not centered in their shutters. | There is a bug that prevents the wavelength correction for offset point sources, as calculated in the wavecorr step, from being propagated to the rectified 2D spectra in the s2d data products. | Reprocess spectra using the new version of the calibration pipeline software (jwst 1.15.0 or later). The wavelength corrections for point sources are now applied correctly and propagated to the resample_spec step. There are no such corrections applied for extended sources, whose wavelength corrections depend on the source geometry. | Updated Operations Pipeline A fix for this issue has been implemented in the jwst calibration software (jwst 1.15.0 and later) and was made available in the Operations Pipeline installed on August 29, 2024. Reprocessing of affected data typically takes 2–4 weeks after the update. |
NIRSpec bright object time series (BOTS)
The information in this table is excerpted from NIRSpec BOTS Known Issues.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-BOTS02: Large number of outlier pixels are identified in the jump step, which leads to lower SNR of the "rateints" products (slope images) per integration. | The jump step default threshold, in combination with the current reference file uncertainties, results in too many pixels being flagged as cosmic rays. | Rerun the jump step with the a rejection_threshold value > 10 (default is 4.0). | N/A | Updated issue None at this time. Updates await investigation into optimal jump thresholds. |
| NS-BOTS03: There is vertical striping across the NIRSpec BOTS rate images (2-D spectra), which appears as negative and/or surplus flux in extracted 1-D spectra (typically with an irregular wavelength-dependent undulation). | Correlated noise from low level detector thermal instabilities (1/f noise) is seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector. | Run NSClean script developed by B. Rauscher on count rate images, using an appropriate mask. Several additional community tools that mitigate 1/f noise are also available. A notebook demonstrating the use of the NSClean algorithm is now available. Additional 1/f cleaning options in the pipeline have been included in jwst 1.16.0 (and later versions), which can now be run on group data in calwebb_detector1 using the clean_flicker_noise step. | N/A | Updated issue NSClean was implemented in the pipeline (jwst 1.13.0 and onwards) as a non-default option. Details on how to invoke NSClean within the science calibration pipeline are provided in the workaround notebook. Investigations are underway to explore whether 1/f cleaning can be turned on by default in the pipeline. |
NS-BOTS04: The extracted 1-D spectrum ("x1d" product) has spurious absorption features, typically one pixel wide. | Pixels marked as "DO_NOT_USE" in the reference data are not included in 1-D extracted spectrum. | Use new science calibration pipeline software (jwst 1.11.0 onwards) to estimate the value for each "DO_NOT_USE" pixel from neighboring 2-D profiles before extracting 1-D spectrum. This is disabled by default; testing has shown improvements in some cases, but degradation in data quality has also been seen in certain circumstances; users should exercise caution if they choose to run the step. | N/A | Updated issue Test that the pixel_replace algorithm works well for NIRSpec fixed slit data. If it does, enable the pixel_replace step by default in the parameter reference file update (estimated for September 2023). Users can enable this step by following commands in this file. The team is evaluating options for further improvements. |
NS-BOTS05: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture. | The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture. | Use only the spectra extracted from the upper dither positions in the affected wavelength range. | N/A | Created issue Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE. |
NS-BOTS06: Fluxes are too low at the red and blue wavelength extremes of extracted 1-D spectra for M- and H-grating observations | The default extract_1d extraction uses a rectangular extraction box that does not follow the curved trace of M- and H-grating data in "calints" files. This causes the 1-D extraction to miss some of the flux at the red and blue wavelength extremes for M-grating data and can miss the red and blue ends of the H-grating traces entirely. | A curved trace can be used for 1-D extraction by providing extract_1d.override_extract1d a modified "extract_1d" parameter reference file that provides a list of polynomial coefficients that define the trace via the src_coeff keyword (see the extract_1d step description for more details). Alternatively see the calwebb_spec2 workaround notebook for BOTS data from JWebbinar 29 for an example of a custom extraction of BOTS data. | N/A | Created issue None at this time. Updates await investigation into potential solutions.
|
| NS-BOTS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. This causes time-series observations (TSO) light curve scatter to increase by orders of magnitude larger than expected. | Some subarrays have no reference pixels, which means bias drifts are not corrected. | Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps. | Updated Operations Pipeline The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. This was implemented in the Operations Pipeline on August 24, 2023. |
| Notable updates | |
|---|---|
| Originally published |
|