Known issues specific to NIRCam imaging data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. 

Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRCam Imaging Calibration Status for an overview of the current astrometric, photometric, and target acquisition accuracy of NIRCam imaging data products.

Artifacts

Information on NIRCam instrument artifacts are available in the main NIRCam Known Issues article.

Pipeline notes

The topics below affect imaging observations and reflect common questions about how to improve the quality of the data from the pipeline. For issues that affect all observing modes, see NIRCam Known Issues.

Background

Offsets in background levels 

In some subarray data, there may be noticeable offsets in the background level from integration to integration even though the data are all at the same pointing. For subarrays, the limited (in the shortwave channel) or lack of (in the longwave channel) reference pixels can cause this effect. To bring the integrations in line with one another, the use of the skymatch step in the stage 3 calibration pipeline is recommended. If you have dedicated background observations, the background subtraction step may also be helpful. More information about stages 2 and 3 background subtraction in the calibration pipeline can be found in the background step documentation. 

Image alignment

The JWST Science Calibration Pipeline is designed to produce fully calibrated, rectified mosaics using a common set of parameters; however, the parameters may not be optimal to produce well-aligned images and mosaics in some cases. As shown on the NIRCam Imaging Calibration Status page, the source position uncertainties in NIRCam data after alignment are typically very small fractions of a pixel. However in some cases, when trying to create an image mosaic, some of the tiles may appear to have WCS values that are offset from their expected locations by a significantly larger amount. Two common reasons behind mosaics that exhibit misalignment in some tiles/detectors are: Having FGS guide on the incorrect guide star for some of the tiles, and small uncertainties in the locations of the detectors relative to one another in the focal plane.

There are parameters within the pipeline that can be updated to tweak and improve the alignment results, as well as community tools for cases where the calibration pipeline is not performing optimally. There are two supported software packages that can be used for image alignment. The first is the tweakreg step within the stage 3 calibration pipeline. This is very similar to the tweakreg step within Drizzlepac. The second is the JWST/HST Alignment Tool (JHAT). 

Small WCS offsets between modules/detectors (milliarcseconds to arcseconds)

There are small (several milliarcseconds at most) uncertainties in the locations of the detectors relative to one another in the focal plane. Generally, the uncertainties between detectors in the same module are quite small. The errors are largest when comparing detectors between the A and B modules (but still no more than several milliarcseconds). These uncertainties can lead to errors in the WCS of a mosaic image at levels that some users may care about. The text below describes one strategy to improve the alignment across the mosaic. The strategy involves running parts of the pipeline separately on data from each module. Note that this general strategy of running the pipeline independently on different subsets of data can be quite useful when attempting to fix various misalignments.

Using Tweakreg

There are essentially two parts to tweakreg: (1) it first does relative alignment across all exposures in the association (using sets of overlapping sources across the various sets of exposures), then (2) it carries out absolute alignment of the entire association of exposures onto an external frame—at this point it considers all the relative alignments between exposures to now be fixed, so it applies the absolute alignment to the entire association of exposures, without further adjusting the relative alignments between exposures.

The relative alignment (part 1) includes these parameters (in the "Object matching parameters" section in the page for the step arguments):

• minobj, searchrad, use2dhist, separation, tolerance, fitgeometry, nclip, sigma

and the absolute alignment (part 2) has a completely independent set of similarly named parameters just with "abs_" prepended, ie:

• abs_refcat, abs_minobj, abs_searchrad, abs_use2dhist, abs_separation, abs_tolerance, abs_fitgeometry, abs_nclip, abs_sigma

The relative alignment (part 1) creates catalogs from the exposures and uses those to do the alignment. When doing this, it assumes all the detectors listed in a given input file are already tied together with accurate relative WCS (i.e., it doesn't try to solve for shifts or rotations of each detector separately). The following approach describes how to improve the alignment in a case where uncertainty between detector locations in modules A and B result in misalignments in the output mosaic image.

If the overall goal is to align to Gaia, the detectors in the shortwave (SW) channel may have too few or no Gaia sources, which is a relatively common issue. However, if the dataset also has longwave (LW) data, the LW data will often have enough Gaia sources due to the larger area of the detectors in the LW channel. Using the SW and LW data together provides a fairly robust approach for improving the alignment with the following recipe:

1. Start with a LW filter that's fairly close to the SW wavelengths (e.g., F277W, but another filter can be used, too) and run tweakreg with the full relative and absolute alignment on each set of LW exposures separately, i.e., module A by itself, and module B by itself. It's possible to do this a few different ways: by running the step in a subdirectory where only the files from a single module are present, or alternatively by giving tweakreg an association containing files only for one module at a time, or as a 3rd option by using the new meta.group_id option in tweakreg to give it files for only one module at a time. This allows tweakreg to accurately place each of module A and module B onto Gaia. After completing that step, stop the calwebb_image3 pipeline at this point before continuing beyond tweakreg by setting skip=True for the subsequent steps in the pipeline. Be sure to set save_results = True in tweakreg so that it produces a set of "*_tweakreg.fits" files. (Note: if the exposures are from different visits, then it may be desirable to run the above for each visit separately by itself).
   
   
2. Produce a full mosaic of all these LW tweakreg files in this filter (i.e., combining all the exposures in module A and B) by rerunning the calwebb_image3 pipeline, giving it all the "*_tweakreg.fits" files from both modules A and B as input: in this rerun, tweakreg would now be skipped by setting skip = True, so that the pipeline can continue directly onto the subsequent steps of outlier rejection, mosaic production, and catalog generation; the resulting LW catalog should now be on the Gaia frame.
   
   
3. Then, run tweakreg on all the other data from the other filters in SW and LW using the same approach of running all the files separately for each module (and again, it may be desirable to run this separately for each visit, if that’s the case). This time, provide the above LW catalog as input for the abs_refcat parameter (i.e., no longer using the Gaia catalog, which may be too sparse for some of the detectors, but rather using the much denser LW catalog produced above, which should be on the Gaia frame). Be sure to stop the pipeline at the end of the tweakreg step by setting skip = True for the subsequent steps, and remember to set save_results = True so that it produces a set of "*_tweakreg.fits" files.

Once all the "*_tweakreg.fits" files have been produced for all the exposures in a given filter, these can then be provided to the calwebb_image3 pipeline for the subsequent steps (i.e., tweakreg would now be skipped by setting skip = True, so that the pipeline can continue directly onto outlier rejection, mosaic production, and catalog generation).

Using the JWST/HST Alignment Tool (JHAT)

For observers who find that tweakreg does not align their images well, it may be helpful to try using the JWST/HST Alignment Tool (JHAT). This tool provides robust WCS alignment of JWST and HST images using Gaia, PS1, DECam, or a custom catalog. JHAT can mitigate some of the common issues with classical HST/JWST image alignment, and functions inside the JWST pipeline analysis flow to improve absolute and relative alignment for the final data products.The JHAT Documentation page contains several examples involving alignment of JWST and HST datasets.

Small WCS offsets between exposures (arcseconds)

When aligning NIRCam mosaics, some of the individual exposures may be well aligned, but others have offsets of several arcseconds from where they should be. These offsets can be present in the "rate.fits" and "cal.fits" files prior to running the Stage 3 pipeline. In cases where some data have a noticeable shift compared to others, it can be that the FGS was guiding on the wrong star when some of the data were acquired. In that case, the FGS pointing is incorrect, which propagates into the reported NIRCam pointing. In these cases, if the pointing is incorrect by several arcseconds, it is possible to correct the pointing and produce a well-aligned mosaic. 

• To correct using tweakreg: try setting the fit_geometry parameter to rscale or general. The value to use depends on the number of sources in the images. rscale will only adjust the rotation and scale of the input images, while general will adjust the shift, rotation, and scale. For a description of these and other tweakreg parameters, see the step arguments page in the tweakreg documentation. general requires more sources (hundreds to thousands) than rscale. This will allow more flexibility in how the images are adjusted/oriented relative to one another. Next increase the values of the separation and tolerance parameters. When attempting to match sources, tolerance is the maximum offset allowed after applying an initial WCS correction to a file. Both are in units of arcseconds, and in general, separation should be ~2x tolerance. Example code to make these changes is shown below:

# call just the tweakreg step and save the results to a fits file
from jwst.tweakreg import TweakRegStep
tweakreg = TweakRegStep.call(asn_file, fitgeometry='general', tolerance=1.0, separation=2.0, save_results=True)
 
# or as part of a call to the stage 3 pipeline:
from jwst.pipeline.calwebb_image3 import Image3Pipeline
params = {'tweakreg': {'fitgeometry': 'general',
                       'tolerance': 1.0,
                       'separation': 2.0
                       }
          }
output = Image3Pipeline.call(asn_file, steps=params, save_results=True)

• To correct with JHAT: call JHAT once using only the data with the bad WCS offset. In this call, set the xshift/yshift parameter values to roughly the values needed to bring the WCS closer to what it should be (or what it is in the data with no offset). In addition, use the --iterate_with_xyshifts flag. Once that is complete, make a second call to JHAT using the data without the WCS offfset. In that case, no xshift/yshift is needed. If those steps are successful, then you can call the stage3 pipeline on all the data together, but be sure to turn off the tweakreg step so that it does not try to re-adjust the WCS of the data.

Large WCS offsets (arcminutes)

For cases where there appear to be some tiles in a mosaic that are arcminutes from where they should be, they may have the incorrect WCS. WCS offsets this large are most likely due to missing engineering data. Check the value of the ENGQLPTG header keyword in your files. If the value is PLANNED then most likely the engineering data needed to calculate the pointing was not found during the initial pipeline run, and therefore the pointing was never updated. To resolve this, try running the "set_telescope_pointing.py" script on each of the files with the bad WCS. This script is part of the jwst calibration package. At the command line (rather than within Python), use: set_telescope_pointing.py <file_uncal.fits>, and replace <file_uncal.fits> with the name of your "uncal" file. It can also be run on "rate" files. This will search the engineering data and update the pointing information in the file. You'll then have to run the corrected "uncal" files through the calibration pipeline.

Offsets between NIRCam and Hubble images

When trying to align NIRCam and HST images (e.g., ACS), users may notice small systematic offsets (< 1 arcsecond) in the WCS coordinates between two image sets. It is possible to update the NIRCam WCS information using the reference HST images with the JHAT tool. The JHAT documentation has a number of examples that should be helpful. If you are happy with the relative alignment of the NIRCam images to one another, then it should be a relatively straight forward process to generate a source catalog by running JHAT on the HST data, and then applying that catalog to the JWST data with another JHAT run.

Aligning data to a reference image

Once a single NIRCam image has been aligned with an HST reference image with JHAT, it is possible to align the remaining NIRCam images with the initial NIRCam image. The path forward depends on your data. If you have a good amount of overlap between the image that you aligned with HST and the other images, then you can treat the former as your reference image and align the others to it using JHAT. See the NIRCam example in the JHAT documentation: Relative Alignment.

If you are working with an extended mosaic where there is not a lot of overlap (or not a lot of sources in the overlap regions) between the NIRCam images, then it might be better to align the images using a catalog, such as GAIA: see Align to Catalog. In this case though, JHAT may adjust the WCS in the image that you aligned to HST. In that case, you might need to go back and align the HST data using GAIA as well.

Resampling

Resampling images onto the same pixel grid

In some cases, observers may wish to output the images of the same field taken in multiple filters onto the same pixel grid. In the resample step in calwebb_image3, users can specify the output WCS using these parameters, which are described in more detail here: Resampling Step Arguments

• pixel_scale (in arcsec)
• rotation (0.0 gives north up)
• crpix [crpix1, crpix2]
• crval [crval1, crval2]
• output_shape [naxis1, naxis2]

When choosing the pixel_scale, note that values of 0.03" and 0.06" are similar in size to the native detector pixel scales for NIRCam SW and LW, respectively. Smaller values, such as 0.02" and 0.04", are examples of choices that sample the native detector pixel scale more finely for SW and LW respectively, to take advantage of subpixel dithering if there are a sufficient number of exposures. Note that smaller pixel scales result in larger files that have more pixels, and will require more memory to process. It is also worth noting that the crpix values are interpreted by Python as 0-indexed, so the resample step will increase their values by 1 when populating the CRPIX1 and CRPIX2 keywords in the FITS header.

Once users have determined the values to use, the parameters can be provided to calwebb_image3 as follows:

from jwst.pipeline.calwebb_image3 import Image3Pipeline  

# Initialize the nested dictionary
params = {} 
params['resample'] = {}

# Add the desired parameters
params['resample']['pixel_scale'] = 0.03
params['resample']['rotation'] = 0.0
params['resample']['crpix'] = [1347.2, 1325.2]
params['resample']['crval'] = [67.014, 21.5439]
params['resample']['output_shape'] = [2628, 2632]

# Call the stage 3 pipeline  
output = Image3Pipeline.call('stage_3_association.json', save_results=True, steps=params)

Alternatively, users can add an output WCS into which resample will drizzle the data using the WCS from the first image as a reference. To do this, users will need to ensure the first image includes the WCS from the pipeline, either by running the image through calwebb_image3 or opening an existing mosaic produced by calwebb_image3. The WCS from that file must be saved as an ASDF file. Then, when running calwebb_image3 on the other filters, users can provide this ASDF file for the output_wcs parameter and specify the same output image size. In this case, it is not necessary to set the other resample step parameters to match the values from the first mosaic, because the file specified in output_wcs should take care of the other keywords. Below is an example of how to do this.

from asdf import AsdfFile 
from jwst import datamodels 
from jwst.pipeline.calwebb_image3 import Image3Pipeline 

# Create the mosaic image for the first filter 
filter1 = Image3Pipeline.call('my_filter1_asn.json, save_results=True) 

# Or open an existing mosaic file: 
filter1 = datamodels.open('my_mosaic_i2d.fits') 

# Pull out the resulting gWCS and save it in an ASDF file 
tree = {"wcs": filter1.meta.wcs} 
wcs_file = AsdfFile(tree) 
outfile = 'my_gwcs.asdf' 
wcs_file.write_to(outfile) 

# Now call the stage 3 pipeline on the files for the next filter, and supply the gWCS file in the input parameters 
params = {} 
params['resample'] = {}
params['resample']['output_wcs'] = wcs_file 

# Set the output shape to match the shape of the first mosaic 
ysize, xsize = filter1.data.shape 
params['resample']['output_shape'] = (xsize, ysize) 

# Call the stage 3 pipeline 
call_output = Image3Pipeline.call('my_filter2_asn.json', save_results=True, steps=params)

Source catalog

Aperture photometry

In some cases, observers may want to compare the aperture and isophotal photometry produced by the source_catalog step in the calwebb_image3. For more information on this, the JWST Data Analysis Tool Notebooks are a helpful resource. In the JWST Data Analysis Tool notebooks, there is an Extended Aperture Photometry notebook that shows an example of working with isophotal photometry. Also, the Point Source Aperture Photometry notebook shows an example of performing aperture photometry, including aperture correction.

Stage 3 processing

Pixel scale and rotation angles

When calling the stage 3 pipeline to make a final mosaic from individual "*cal.fits" images, some observers may wish to specify the final pixel scale and rotation angle of the mosaic. For calwebb_image3, you can control the final pixel scale and rotation using input parameters, just like with Drizzlepac. The parameters you need to use are pixel_scale and rotation in the resample step. These are described in the documentation for the resample step.

# call the entire pipeline while updating the resample parameters
from jwst.pipeline.calwebb_image3 import Image3Pipeline
params = {'resample': {'pixel_scale': '0.02',
                       'rotation': 0.0
                       }
          }
result = Image3Pipeline.call('my_asn.json', steps=params, save_results=True)

Cosmic ray flagged data data products

The stage 3 pipelines (calwebb_image3 and calwebb_spec3) include the outlier detection step, which finds and flags outlier pixel values. The results of this process have the same format and content as the "cal.fits" data products, but with cosmic ray flagging, and are saved to a ".crf.fits" data product. These files are one of the default outputs when running calwebb_image3.  They represent the final state of the individual exposures, including the most up to date WCS and DQ flags. In short, the pipeline is saving the data that go into creating the final mosaic. Note that when running the individual steps of the pipeline, rather than the pipeline itself, no ".crf" files are saved. However, in this case, if you save the output of the outlier_detection step, the "*outlierdetectionstep.fits" files that are produced are the same as the ".crf" files.

Summary of common issues and workarounds

The sections above provide detail on some of the known issues affecting NIRCam imaging data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.

References

Rest, A., Roberts-Pierel, J., Correnti, M., Hilbert, B., Canipe, A., and Sunnquist, B., vol. 55, no. 2, 2023.
JWST/HST Alignment Tool (JHAT)