Known Issues with JWST Data

Specific artifacts that users are likely to encounter in their JWST data are provided in this series of articles. In them, you will find notes on how to best run individual pipeline steps offline to optimize data quality for different kinds of science, and summaries of some common workarounds to existing problems.

On this page

Despite the generally high calibration quality of JWST data, there are nonetheless a series of known issues with these data products that science users should be aware of. These range from detector artifacts (which cannot be fixed) to reference file and pipeline software problems (which can be improved over time) to optional offline processing improvements that can be made for some science cases.

The articles are structured as follows:

  • This page highlights known issues affecting all JWST observing modes.
  • Instrument-specific pages highlight known issues affecting MIRI, NIRCam, NIRISS, and NIRSpec data.
  • For each instrument-specific page, child pages discuss known issues affecting specific instrument modes (e.g., NIRcam imaging, MIRI MRS, etc.).
  • An overview of frequently-encountered issues and workarounds across all JWST observing modes are compiled in a high-level summary page.

Please also refer to JWST Calibration Status for an overview of the general calibration quality of JWST data (photometric calibration, wavelength calibration, etc.).



Artifacts

Cosmic ray shower and snowball artifacts

Cosmic rays typically produce small artifacts in detector data that are easily handled by the JWST pipeline, but certain kinds of cosmic rays can produce large artifacts that cover hundreds or thousands of detector pixels. See Shower and Snowball Artifacts

1/f noise

JWST detectors exhibit different types of frame-to-frame noise produced by the readout electronics. One of particular interest for observations of faint targets or time-series observations (see also JWST Time-Series Observations Noise Sources) is the so called 1/f noise in near-IR data. The most obvious visual feature of this noise is an apparent "banding" of the background pixels in a frame/group. Removing the impact of 1/f noise on the data is quite complex given the stochastic nature of the effect (every frame will have its unique signature—however, they will all share similar statistical properties). While the reference pixel correction helps diminish its effect in some cases, in general, extra steps may need to be taken in an attempt to correct for it.

At present, there is no common way of treating this effect across all three near-IR detectors. For information on 1/f correction in NIRCam data, see NIRCam 1/f Noise Removal Methods; for NIRSpec see NIRSpec Known Issues.

Note that this 1/f noise is primarily a product of the micro-electronic SIDECAR ASICs in use for the NIR detectors. The full size electronics in use by the MIRI instrument generally have more stable voltage and current supplies, and the MIR detectors therefore do not see significant 1/f noise.



Pipeline notes

Incorrect world coordinates

See also: JWST Pointing Performance, JWST Attitude Control Subsystem

Words in bold are GUI menus/
panels or data software packages; 
bold italics are buttons in GUI
tools or package parameters.

When observing without target acquisition, the placement of a given scientific target can be limited by multiple factors, including both the JWST blind pointing uncertainty and any uncertainty in the target coordinates themselves.

Even when using target acquisition to place a source precisely at the intended location within a given instrument however, the world coordinates embedded into the final data products by the JWST pipeline are subject to an observatory-wide WCS accuracy error. The magnitude of this error varies by field and by instrument, and is typically largest for the MIRI MRS (1-σ radial accuracy about 0.3", with known examples over 1.0"). These errors are driven by a combination of uncertainties in the guide star catalog, misidentified guide stars during observations, and uncertainty in the spacecraft roll angle as measured from the star trackers.

This in turn can cause problems with mosaic tile alignment, registration against data from other observatories, and spectral extraction from fixed target locations using sky coordinates.

Mitigations for this problem depend on the instrument mode in use. Imaging modes can typically use the tweakreg step to re-register coordinates against known sources within the field of view, while IFU observations can benefit from auto-centroiding to find the appropriate location for point source spectral extraction.  See the individual instrument and mode known issues articles for further details.

In the long term, work is ongoing to improve guiding performance.

Pointing jitter or drift

See also: JWST Pointing PerformanceJWST Communications SubsystemJWST Attitude Control Subsystem

The JWST calibration pipeline typically assumes that the pointing of the telescope is perfect and constant over the course of an observation. In reality, the initial target placement may be imperfect, and over many hours the pointing may be subject to drifts, jitter, or jumps. The high gain antenna repointing, for example, has been observed during commissioning to have some impact on the telescope pointing when it is moved. 

Such drift may have particular consequences for time-series observations. 

For time-series imaging, the automated photometry step in calwebb_tso3 assumes a fixed location of the target in the field of view. If this location changes significantly over the course of the time-series observation, the photometry measurement will be performed off-center and return inaccurate results. 

For time-series spectroscopy, the wavelength calibration and photometric calibration factors are applied to the data assuming perfect and constant target placement in the field. Any drifting behavior or jitter, therefore, might introduce systematic noise in the final time series. There is currently no mitigation for this in the calibration pipeline; any pointing changes must therefore be measured and corrected manually in the time series. A suggested workaround is to skip the photometric calibration step by modifying the execution parameters for the calwebb_spec2 stage since for many science needs it is the relative variations that are of most interest. Further pipeline work in this area is planned.

In general, time-series observations may find it useful to monitor the JWST guide star data which can be retrieved via MAST, and use it to study any significant pointing aberrations that might be impacting a given TSO. Details on how to perform this data retrieval can be found in JWST Time-Series Observations Noise Sources.

Time stamps across the JWST detectors

The detectors used by JWST instruments do not read all their pixels at the same time. While this is typically irrelevant for most science cases, it can be important for high-cadence time-series observations.

For the near-infrared detectors, the reading process controlled by the SIDECAR ASIC sequentially reads pixels in the "fast-read" direction. In NIRISS/SOSS frames, the reading process starts in one of the corner pixels and moves along the fast-read direction, reading one pixel at a time at a cadence of 10 microseconds per pixel. When all the pixels in a given "fast-read" column are read, the detector has a wait time of 120 microseconds before moving to the next one. This process is repeated until the entire subarray is read, after which extra wait time (which depends on the exact subarray configuration) is spent before considering the frame "read". Note that for this NIRISS/SOSS example, this means not all columns are read at the same times, but instead, some are read earlier than the rest. In practice, for NIRISS/SOSS, this means that the longer wavelengths are effectively read by the detector at different times than the shorter wavelengths—the difference being about 5 s in total per frame. This is currently not accounted for in the pipeline. Users should study whether this offset might impact their science cases, and consider making corrections to their observations based on this so every wavelength has the correct time stamp associated with it.

The time constants may be different for each instrument, and dependent on the subarray and read mode. For more detail see the NIRCAM, NIRISS, NIRSpec, and MIRI detector readout pages.



Summary of common issues and workarounds

The tables below provide summaries of some of the most likely issues for users to encounter when working with JWST data in general, along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build. For a more extensive list see the individual known issues articles for different instruments and observing modes.

General issues

SymptomsCauseWorkaroundFix buildMitigation plan
GI02: Embedded world coordinate system (WCS) in JWST data products is incorrect.

Errors in the guide star catalog, misidentified guide stars, and uncertainties in the spacecraft roll angle result in errors in the WCS of pipeline data products even when target acquisition was performed to place science targets in the correct location. Typical errors are a few tenths of an arcsec, with some cases that are greater than 1 arcsec.

The workaround depends on instrument and mode. Also see issue NC-I01. 

N/A

Updated issue

Improve accuracy of the guide star catalog. This is a long-term project. (Updated "Workaround" to mention NC-I01)

GI03: Data contain shower and snowball artifacts.

These are caused by large cosmic ray impacts.

The calwebb_detector1 pipeline includes a snowball/shower correction.  This correction is run by default for many observing modes, but is turned off in some others until testing indicates satisfactory performance in corner cases.

There is no workaround that works for all science cases.

The correction is not recommended for NIRISS SOSS or AMI,  MIRI coronagraphic data, and data with 1–4 groups.

For general science cases, users can re-run the pipeline calwebb_detector1 with the jump step parameters set:
   find_showers = True  (For MIRI)
   expand_large_events = True (for NIR instruments)


N/A

Updated issue

Snowball/shower correction in the jump detection step of calwebb_detector1 will be implemented in the pipeline and enabled via delivery of new parameter reference files for each instrument, as they become available.  As of Build 10.2 automated processing occurs for most NIR modes and MIRI imaging.

See the section titled "Large Events (Snowballs and Showers)" in the JumpStep documentation, and  the Shower and Snowball Artifacts article, for more information on the status of these corrections.

GI04: NIR instruments only: There is large-scale striping (horizontal for NIRCam, vertical for NIRISS and NIRSpec) across the field and not fully removed in the reference pixel subtraction. Note that IRS2 readout for NIRSpec substantially mitigates this behavior.1/f noise from the SIDECAR ASICs (detector readout electronics) causes this effect.

There are several community tools available that are designed to remove 1/f noise. 

N/A

Updated issue

A mitigation plan is being developed to determine which if any tool to integrate into the pipeline.


GI05: Stage 3 processing of large imaging mosaics can take a longer than the normal amount of time to process.Unknown.

None.

N/A

Created issue

Updates are planned for the methods used in the tweakreg and resample steps to make them more efficient.

GI06: World Coordinate System (WCS) in pure parallel data products is incorrect by amounts that are different for every dither position.In pure parallel data, the values of the WCS-related header keywords are currently derived using a “coarse” algorithm, because the guide star information is currently not being transferred from the headers of prime exposures to those of the associated pure parallel exposures. Typical errors are of order 0.1 arcsec. While these errors can be benign for imaging data (since the spatial offsets can be corrected in the tweakreg step of the calwebb_image3 pipeline), they are problematic for pure parallel WFSS-mode grism exposures, because the placement of spectral extraction boxes by the calwebb_spec2 pipeline relies on the WCS information in the data headers.

The WCS in pure parallel data products can be corrected by running a script in the JWST caveat examples github repository

Created issue

Fixed in build 10.2 by updates to the way that WCS header keywords are populated for pure parallel data.

GI01: TARG_RA and TARG_DEC in the FITS primary header are not at the epoch of the JWST exposure. This is one reason the 1-D spectral extraction aperture can be offset from the target location in the 2-D extracted spectrum image (see relevant instrument modes).

Initially, science data processing was not applying proper motion to the target coordinates specified by the user (PROP_RA, PROP_DEC). After a update, science data processing began applying a proper motion correction that was too small by a factor of 0.36533.

Download uncalibrated data. Update TARG_RA and TARG_DEC (see workaround). Rerun calibration pipeline.

Updated Operations Pipeline

Proper motion was applied correctly. STScI reprocessed affected data products with an updated Operations Pipeline installed on August 24, 2023. Reprocessing of affected data typically takes 2-4 weeks after the update.

General issues for time-series observations

SymptomsCauseWorkaroundFix buildMitigation Plan

GI-TS01: For time-series data (for all instruments), FITS primary header keywords are different from the "INT_TIMES" extension. Particularly, this concerns the start/end times (BSTRTIME and BENDTIME) and the barycentric correction (BARTDELT) keyword.

"INT_TIMES" are based on the group times directly read into the engineering data. This is not the case with the header keywords, which do not account for electronic shifts on the reading of the data.

Use "INT_TIMES".

Updated Operations Pipeline

A change to the JWST Science Data Processing subsystem to correctly compute the barycentric and heliocentric time, and JWST barycentric position keywords was part of the updated Operations Pipeline, installed on August 24, 2023. STScI reprocessed affected data products, which  typically takes 2–4 weeks after the update.




Notable updates
Originally published

 



  PageInfo Page Properties table


CategoryDataArticles
OwnerDavid Law 
PRdate

Workflow

Comments

This table is used with page properties reports to collect information about pages. Fields shaded red – Category and Workflow – should not be edited. Feel free to use other fields for your records/notes. Its use is optional.

  1. Category - article type, please do not edit it. 
  2. Owner: person responsible for page updates. (Use "@" to enter a name)
  3. PRdate: Date this page was last set to Publication Ready. Please do not edit. 
  4. Workflow: status appears automatically. Please do not edit. 
  5. Comments: brief minor comments, one per bullet. For major issues, create a JDOXCM ticket.