JWST Imaging Pipeline Caveats
Content Migration
WARNING: This page has been deprecated and the content migrated to Known Issues with JWST Data. Please update your links accordingly.
JWST Science Calibration Pipeline features and potential caveats specific to various kinds of JWST imaging data are described in this article.
On this page
Summary of imaging pipeline issues
MIRI imaging
The information in this table about MIRI imaging pipeline issues is excerpted from Known Issues with JWST Data Products.
| Symptoms | Cause | Workaround | Mitigation Plan |
|---|---|---|---|
MR-I02: The "cal" data product has "tree ring" like pattern. | The "tree ring" pattern of curves in MIRI images is a residual detector effect that is not removed in all cases by the reference files. | Creating and subtracting a median background can remove residual detector patterns in the background like the 'tree rings'. This notebook demonstrates how it can be done. | Created issue Reference file and pipeline updates may be available at a future date to remove residual patterns such as the "tree rings" from all data, but there is no timeline on when this might be accomplished. |
MR-I03: The photon count rate and derived flux are lower than predicted at wavelengths between 12.8 and 25.5 µm, with the effect increasing with wavelength (see JWST Observer). | The MIRI imager sensitivity at long wavelengths is decreasing with time. The root cause of this issue is still under investigation. Regular monitoring observations are being taken with the MIRI imager to measure the photometric response and to characterize the temporal trend. | New "photom" files were delivered on September 15, 2023 and there are 2 possible workarounds:
| Updated issue In the short term, new "photom" reference files tailored for specific epochs have been provided in CRDS. STScI will update the pipeline to include a new time-dependent throughput correction, expected to occur in February 2024. Both mitigation strategies will require a reprocessing of the data. |
MR-I01: Image has residual background. | The backgrounds for longer wavelength images can be quite high, and difficult to work with if dedicated background images are not taken and subtracted off in the pipeline. | Users can create a median sky (background image) and subtract it off for themselves if there is no extended source such as a galaxy or nebula in the image. This is most useful for long wavelength imaging, but can also be used for shorter wavelength images. This notebook demonstrates how it can be done. | Created issue This is not something that can (or should) be changed in the pipeline, and will need to be done as an offline step as shown in the notebook. |
NIRCam imaging
The information in this table about NIRCam imaging pipeline issues is excerpted from Known Issues with JWST Data Products.
| Symptoms | Cause | Workaround | Mitigation Plan |
|---|---|---|---|
| NC-I01: Misalignment of mosaic tiles in some of the stage 3 data products | This is usually due to an insufficient number of stars for alignment using the default calwebb_image3 pipeline parameters. | Users can adjust the parameters to the tweakreg step in calwebb_image3 to find more stars across the field for alignment. This notebook shows how to adjust the tweakreg parameters and rerun the stage 3 pipeline. Users can also choose to align to Gaia DR3 or input a custom reference catalog to tweakreg. The community tool JWST/HST Alignment Tool (JHAT) also produces improved alignment. | Updated issue Reprocess data with an enhanced calibration reference file (distortion) in CRDS, which may improve alignment in some cases. An update is planned for early 2024. Reprocessing of old data typically takes 2–4 weeks after the update. |
| NC-I02: Some of the mosaic tiles in the stage 3 products are well aligned, but a subset of the tiles are offset. | Guiding on different stars between mosaic tiles can sometimes cause a misalignment. | In calwebb_image3, try adjusting the tweakreg fit_geometry parameter to rscale or general, which provides more flexibility in how the images are adjusted/oriented. Also try adjusting the separation and tolerance parameters to provide more stellar matches between images. Alternatively, run JHAT separately on the well-aligned and poorly-aligned data. Then, feed all data into calwebb_image3. Be sure to turn off the tweakreg step since the data have already been aligned. | Created issue Mitigation is not yet scheduled due to higher priority issues. |
| NC-I03: There are large scattered light features on the images of some NIRCam detectors. | There are several NIRCam scattered light artifacts. The most common are claws and wisps, which are caused by light entering directly through the aft optic system (AOS) mask, located in front of the JWST tertiary mirror, without first bouncing off the primary and secondary mirror. | For claws: Avoid position angles that place bright stars in the susceptibility zone. For wisps: Wisps can be subtracted using templates, available to download here: NIRCam Claws and Wisps | Created issue For claws: The mitigation plan for removing claws from data is not yet scheduled. However, the NIRCam team is screening all programs to minimize the risk of claws before programs are observed. For wisps: A GO calibration program (PID 3905) aims to improve the wisp templates. |
| NC-I04: An excessive number of pixels are flagged as outliers in some subarray data. | Some imaging subarrays do not have reference pixels on all sides, especially the extended source subarrays in the long wavelength channel. Without a reference pixel correction, the data become noisier and the jump step in calwebb_detector1 sometimes identifies too many pixels as outliers. | Rerun the jump detection step in calwebb_detector1 with an increased rejection_threshold (default is 4.0). | Updated issue The jump step algorithm and default parameters are continually being examined and optimized; improvements are expected in future builds (winter 2023 and beyond). |
| NC-I05: Running tweakreg on exposures with multiple detectors or modules results in misaligned mosaic tiles and offset WCSs. | Tweakreg groups together data for all detectors for a given exposure using information in the Science Instrument Aperture File (SIAF) to determine positions of the detectors relative to one another. It then finds and matches sources working one exposure (but multiple detectors) at a time. In some cases, tweakreg does not do a good job of aligning exposures for all detectors with the sources in the field due to uncertainties in the locations of the detectors within the focal plane at the level of a few pixels. The uncertainty is largest between the A and B modules, and smaller between the detectors within a given module. | Use JHAT (which does not do the detector grouping) rather than tweakreg, or call tweakreg separately for each module and align to GAIA, so that the two modules are both shifted to the appropriate WCS. Another option is to change the metadata in the files such that each detector or module has its own “exposure number”, since that is what tweakreg uses to find and group detectors. | Created issue Updates to the NIRCam detector positions in SIAF coming this winter should reduce the uncertainties down to a small fraction of a pixel. More optimal tweakreg step parameters for the Operations Pipeline are still under investigation. |
NIRISS imaging
The information in this table about NIRISS imaging pipeline issues is excerpted from Known Issues with JWST Data Products.
| Symptoms | Cause | Workaround | Mitigation Plan |
|---|---|---|---|
NR-I01: When the peak pixel of a star (or other compact object) reaches beyond ~25,000 e- in an integration (after bias and dark subtraction and linearity correction), it starts "spilling" charge to its neighboring pixels. This causes an effective "widening" of the PSF or charge distribution, as well as flux loss for those objects in the combined, resampled products of the calwebb_image3 stage of the Science Calibration Pipeline. This effect is strongest for the most undersampled modes (i.e., filters with pivot wavelength <= 2 μm). | This is due to the so-called "brighter-fatter effect" (BFE) that affects near-IR H2RG detectors, in combination with the current way "jumps" are detected and dealt with in the calwebb_detector1 pipeline stage, and how the latter affects image combination in the calwebb_image3 pipeline stage. | No efficient workaround is available at present. | Updated Operations Pipeline Applied the new Science Calibration Pipeline step undersampling_correction in the calwebb_detector1 stage. STScI is reprocessing affected data products with an updated Operations pipeline, installed on December 5, 2023. Reprocessing of affected data typically takes 2–4 weeks after the update. |
About the JWST imaging pipeline
See also: Getting Started with JWST Data, file header contents
Several instruments on JWST provide imaging data of various kinds. For an overview of imaging, refer to the materials and video excerpts from JWebbinar 13 (open the "Materials and Videos" tab, and scroll down to JWebbinar 13, "CEERS NIRCam and MIRI Imaging." Or go directly to the CEERS Overview video on the JWST Observer YouTube channel).
JWST imaging data are processed by the JWST Science Calibration Pipeline. As with other instrument modes, the pipeline is run with various default parameters that are generally appropriate and produce science quality products, but may or may not be optimal for a particular science case. Additionally, imaging modes are often used to produce mosaics of areas larger than a single field of view of the instrument in question, which produces additional complexity for aligning and calibrating the data.
Data products
Software documentation outside JDox: Science Products
The current defaults for rejection of outliers/bad pixels and cosmic rays are based on what has been learned from the detector performance to date, and are aimed at providing satisfactory rejection for most datasets. However, users may want to try non-default parameters to fine-tune this step for specific types of data.
The current defaults for image alignment/tweakreg and catalog creation are aimed at providing satisfactory results for most datasets, but they could be fine-tuned for specific types of data and run offline.
Mosaics
Image mosaics in different filters are not necessarily aligned on the same x,y pixel grid, but the RA, Dec astrometry is expected be consistent for the same source in different filters.
Also, mosaics are created with a pixel scale equal to the detector pixel size, so observations that take advantage of sub-pixel dithering might benefit from being processed offline with a finer pixel scale.