Known issues specific to NIRCam imaging data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. 

Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRCam Imaging Calibration Status for an overview of the current astrometric, photometric, and target acquisition accuracy of NIRCam imaging data products.

Artifacts

Information on NIRCam instrument artifacts are available in the main NIRCam Known Issues article.

Pipeline notes

The topics below affect imaging observations and reflect common questions about how to improve the quality of the data from the pipeline. For issues that affect all observing modes, see NIRCam Known Issues.

Background

Offsets in background levels 

In some subarray data, there may be noticeable offsets in the background level from integration to integration even though the data are all at the same pointing. For subarrays, the limited (in the shortwave channel) or lack of (in the longwave channel) reference pixels is to blame for this effect. To bring the integrations in line with one another, performing a background subtraction is recommended. More information about stages 2 and 3 background subtraction in the calibration pipeline can be found in the background step documentation. 

Image alignment

Offset WCS between mosaic tiles

When trying to create an image mosaic, some of the tiles may appear to have a WCS that seems to be off. The JWST Science Calibration Pipeline is designed to produce fully calibrated, rectified mosaics using a common set of parameters; however, the parameters may not be optimal to produce well-aligned images and mosaics in some cases. In some instances, this may be due to guiding on the incorrect guide star for some of the tiles. 

There are parameters within the pipeline that can be updated to tweak and improve the alignment results, as well as community tools for cases where the calibration pipeline is not performing optimally. There are two supported software packages that can be used for image alignment. The first is the tweakreg step within the stage 3 calibration pipeline. This is very similar to the tweakreg step within Drizzlepac. The second is the JWST/HST Alignment Tool (JHAT). 

Using Tweakreg

There are essentially two parts to tweakreg: (1) it first does relative alignment across all exposures in the association (using sets of overlapping sources across the various sets of exposures), then (2) it carries out absolute alignment of the entire association of exposures onto an external frame—at this point it considers all the relative alignments between exposures to now be fixed, so it applies the absolute alignment to the entire association of exposures, without further adjusting the relative alignments between exposures.

The relative alignment (part 1) includes these parameters (in the "Object matching parameters" section in the page for the step arguments):

• minobj, searchrad, use2dhist, separation, tolerance, fitgeometry, nclip, sigma

and the absolute alignment (part 2) has a completely independent set of similarly named parameters just with "abs_" prepended, ie:

• abs_refcat, abs_minobj, abs_searchrad, abs_use2dhist, abs_separation, abs_tolerance, abs_fitgeometry, abs_nclip, abs_sigma

The relative alignment (part 1) creates catalogs from the exposures and uses those to do the alignment. When doing this, it assumes all the SCAs in a given exposure are already tied together with accurate relative WCS (i.e., it doesn't try to solve for shifts or rotations of each SCA separately). Within each SW module, the separate SCAs generally do seem to be quite well aligned (to a few mas or better), but modules A and B may still have some noticeable residual differences (which are currently being worked on and will be improved in future updates to the instrument WCS in the operational archive); the following approach describes how to improve this alignment.

If the overall goal is to align to Gaia and some SCAs in the shortwave (SW) channel have too few or no Gaia sources (which can happen quite regularly), then if the dataset also has longwave (LW) data that has enough Gaia sources (more likely, given the larger area of the SCAs in the LW channel), then a fairly robust approach to provide improved alignment that has been used in a significant number of JWST programs so far is the following recipe:

1. Start with a LW filter that's fairly close to the SW wavelengths (e.g., F277W, but another filter can be used, too) and run tweakreg with the full relative and absolute alignment on each set of LW exposures separately, i.e., module A by itself, and module B by itself. It's possible to do this a few different ways: by running the step in a subdirectory where only the files from a single module are present, or alternatively by giving tweakreg an association containing files only for one module at a time, or as a 3rd option by using the new meta.group_id option in tweakreg to give it files for only one module at a time. This allows tweakreg to accurately place each of module A and module B onto Gaia. After completing that step, stop the calwebb_image3 pipeline at this point before continuing beyond tweakreg by setting skip=True for the subsequent steps in the pipeline. Be sure to set save_results = True in tweakreg so that it produces a set of "*_tweakreg.fits" files. (Note: if the exposures are from different visits, then it may be desirable to run the above for each visit separately by itself).
   
   
2. Produce a full mosaic of all these LW tweakreg files in this filter (i.e., combining all the exposures in module A and B) by rerunning the calwebb_image3 pipeline, giving it all the "*_tweakreg.fits" files from both modules A and B as input: in this rerun, tweakreg would now be skipped by setting skip = True, so that the pipeline can continue directly onto the subsequent steps of outlier rejection, mosaic production, and catalog generation; the resulting LW catalog should now be on the Gaia frame.
   
   
3. Then, run tweakreg on all the other data from the other filters in SW and LW using the same approach of running all the files separately for each module (and again, it may be desirable to run this separately for each visit, if that’s the case). This time, provide the above LW catalog as input for the abs_refcat parameter (i.e., no longer using the Gaia catalog, which may be too sparse for some of the detectors, but rather using the much denser LW catalog produced above, which should be on the Gaia frame). Be sure to stop the pipeline at the end of the tweakreg step by setting skip = True for the subsequent steps, and remember to set save_results = True so that it produces a set of "*_tweakreg.fits" files.

Once all the "*_tweakreg.fits" files have been produced for all the exposures in a given filter, these can then be provided to the calwebb_image3 pipeline for the subsequent steps (i.e., tweakreg would now be skipped by setting skip = True, so that the pipeline can continue directly onto outlier rejection, mosaic production, and catalog generation).

Using the JWST/HST Alignment Tool (JHAT)

For observers who find that tweakreg does not align their images well, it may be helpful to try using the JWST/HST Alignment Tool (JHAT). This tool provides robust WCS alignment of JWST and HST images using Gaia, PS1, DECam, or a custom catalog. JHAT can mitigate some of the common issues with classical HST/JWST image alignment, and functions inside the JWST pipeline analysis flow to improve absolute and relative alignment for the final data products.The JHAT Documentation page contains several examples involving alignment of JWST and HST datasets.

Offsets (arcseconds) between exposures

When aligning NIRCam mosaics, some of the individual exposures may be well aligned, but others have offsets of several arcseconds from where they should be. These offsets can be present in the "cal.fits" files prior to running the Stage 3 pipeline. In cases where some data have a noticeable shift compared to others, it can be that the FGS was guiding on the wrong star when some of the data were acquired. In that case, the FGS pointing is incorrect, which propagates into the reported NIRCam pointing. In these cases, if the pointing is incorrect by several arcseconds, it is possible to correct the pointing and produce a well-aligned mosaic. 

• To correct using tweakreg: try setting the fit_geometry parameter to rscale or general. The value to use depends on the number of sources in the images. rscale will only adjust the rotation and scale of the input images, while general will adjust the shift, rotation, and scale. For a description of these and other tweakreg parameters, see the step arguments page in the tweakreg documentation. general requires more sources (hundreds to thousands) than rscale. This will allow more flexibility in how the images are adjusted/oriented relative to one another. Next increase the values of the separation and tolerance parameters. When attempting to match sources, tolerance is the maximum offset allowed after applying an initial WCS correction to a file. Both are in units of arcseconds, and in general, separation should be ~2x tolerance. Example code to make these changes is shown below:

# call just the tweakreg step and save the results to a fits file
from jwst.tweakreg import TweakRegStep
tweakreg = TweakRegStep.call(asn_file, fitgeometry='general', tolerance=1.0, separation=2.0, save_results=True)
 
# or as part of a call to the stage 3 pipeline:
from jwst.pipeline.calwebb_image3 import Image3Pipeline
params = {'tweakreg': {'fitgeometry': 'general',
                       'tolerance': 1.0,
                       'separation': 2.0
                       }
          }
output = Image3Pipeline.call(asn_file, steps=params, save_results=True)

• To correct with JHAT: call JHAT once using only the data with the bad WCS offset. In this call, set the xshift/yshift parameter values to roughly the values needed to bring the WCS closer to what it should be (or what it is in the data with no offset). In addition, use the --iterate_with_xyshifts flag. Once that is complete, make a second call to JHAT using the data without the WCS offfset. In that case, no xshift/yshift is needed. If those steps are successful, then you can call the stage3 pipeline on all the data together, but be sure to turn off the tweakreg step so that it does not try to re-adjust the WCS of the data.

Large WCS offsets (arcminutes)

For cases where there appear to be some tiles in a mosaic that are arcminutes from where they should be, they may have the incorrect WCS. WCS offsets this large are most likely due to missing engineering data. Check the value of the ENGQLPTG header keyword in your files. If the value is PLANNED then most likely the engineering data needed to calculate the pointing was not found during the initial pipeline run, and therefore the pointing was never updated. To resolve this, try running the "set_telescope_pointing.py" script on each of the files with the bad WCS. This script is part of the jwst calibration package. At the command line (rather than within Python), use: set_telescope_pointing.py <file_uncal.fits>, and replace <file_uncal.fits> with the name of your "uncal" file. This will search the engineering data and update the pointing information in the file. You'll then have to run the corrected "uncal" files through the calibration pipeline.

Offsets between NIRCam and Hubble images

When trying to align NIRCam and HST images (e.g., ACS), users may notice small systematic offsets (< 1 arcsecond) in the WCS coordinates between two image sets. It is possible to update the NIRCam WCS information using the reference HST images with the JHAT tool. The JHAT documentation has a number of examples that should be helpful. If you are happy with the relative alignment of the NIRCam images to one another, then it should be a relatively straight forward process to generate a source catalog by running JHAT on the HST data, and then applying that catalog to the JWST data with another JHAT run.

Aligning data to a reference image

Once a single NIRCam image has been aligned with an HST reference image with JHAT, it is possible to align the remaining NIRCam images with the initial NIRCam image. The path forward depends on your data. If you have a good amount of overlap between the image that you aligned with HST and the other images, then you can treat the former as your reference image and align the others to it using JHAT. See the NIRCam example in the JHAT documentation: Relative Alignment.

If you are working with an extended mosaic where there is not a lot of overlap (or not a lot of sources in the overlap regions) between the NIRCam images, then it might be better to align the images using a catalog, such as GAIA: see Align to Catalog. In this case though, JHAT may adjust the WCS in the image that you aligned to HST. In that case, you might need to go back and align the HST data using GAIA as well.

Source catalog

Aperture photometry

In some cases, observers may want to compare the aperture and isophotal photometry produced by the source_catalog step in the calwebb_image3. For more information on this, the JWST Data Analysis Tool Notebooks are a helpful resource. In the JWST Data Analysis Tool notebooks, there is an Extended Aperture Photometry notebook that shows an example of working with isophotal photometry. Also, the Point Source Aperture Photometry notebook shows an example of performing aperture photometry, including aperture correction.

Stage 3 processing

Pixel scale and rotation angles

When calling the stage 3 pipeline to make a final mosaic from individual "*cal.fits" images, some observers may wish to specify the final pixel scale and rotation angle of the mosaic. For calwebb_image3, you can control the final pixel scale and rotation using input parameters, just like with Drizzlepac. The parameters you need to use are pixel_scale and rotation in the resample step. These are described in the documentation for the resample step.

# call the entire pipeline while updating the resample parameters
from jwst.pipeline.calwebb_image3 import Image3Pipeline
params = {'resample': {'pixel_scale': '0.02',
                       'rotation': 0.0
                       }
          }
result = Image3Pipeline.call('my_asn.json', steps=params, save_results=True)

Cosmic ray flagged data data products

The stage 3 pipelines (calwebb_image3 and calwebb_spec3) include the outlier detection step, which finds and flags outlier pixel values. The results of this process have the same format and content as the "cal.fits" data products, but with cosmic ray flagging, and are saved to a ".crf.fits" data product. These files are one of the default outputs when running calwebb_image3.  They represent the final state of the individual exposures, including the most up to date WCS and DQ flags. In short, the pipeline is saving the data that go into creating the final mosaic. Note that when running the individual steps of the pipeline, rather than the pipeline itself, no ".crf" files are saved. However, in this case, if you save the output of the outlier_detection step, the "*outlierdetectionstep.fits" files that are produced are the same as the ".crf" files.

Summary of common issues and workarounds

The sections above provide detail on each of the known issues affecting NIRCam imaging data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.

References

Rest, A., Roberts-Pierel, J., Correnti, M., Hilbert, B., Canipe, A., and Sunnquist, B., vol. 55, no. 2, 2023.
JWST/HST Alignment Tool (JHAT)