This article provides information about the performance of community tools that have been developed to mitigate 1/f detector noise in NIRCam data.

Starting with Build 11.1, the JWST Science Calibration pipeline contains a new step to mitigate 1/f noise called clean_flicker_noise, which is currently turned off by default. The performance of the new step for NIRCam data is documented in the NIRCam 1/f Noise Removal Methods article. However, the strategy to remove 1/f noise may be dependent on the science scene, and in some cases, users may wish to try other options or tools. As a service to JWST Observers, publicly available and documented tools developed by the community to assist with 1/f noise mitigation are listed below. These tools are not supported directly by STScI and help for these tools is not available through the JWST Help Desk.

---

Alternative packages for removing 1/f noise

To date, various external software packages have been developed to identify and remove 1/f noise from JWST NIRCam imaging data. A non-comprehensive list of such packages that can be run on "*rate.fits" or "*cal.fits" files is listed in Table 1.

Table 1. 1/f-noise mitigation packages developed in Python for JWST imaging data

"Columns" and "Rows" refer to the fast and slow readout directions, respectively, though these directions may be swapped for different observing modes or instruments.

---

Data

The JWST NIRCam imaging mode is employed for a wide range of astronomical scenes, including sparse extragalactic fields, dense star clusters, and extended sources, such as diffuse and planetary nebulae. A representative set of public data is considered for various types of scenes, as summarized in Table 2 and Figure 2. All images have been calibrated using the STScI pipeline and obtained from the MAST archive. The pipeline processing includes the 1/f correction step using reference pixels; these are insufficient, leaving significant striping visible in the images.

Table 2. Data employed in our comparative analysis

---

Results

All packages were run on the same data set to generate 1/f corrected images. Sources were detected and masked for codes allowing that option. All results are shown below on linear scales, clipped to the same flux levels.

It must be emphasized that most methods were not designed for these general cases. Some assume sparse scenes with "blank" data regions available to measure the 1/f noise. In these cases, all of the methods perform reasonably well. When the image is more crowded or full, most methods do not perform well, oversubtracting the background and/or introducing discontinuities. This can impact the accuracy of absolute flux measurements.

The parameters of some of the packages can be optimized depending on the scene to be corrected. An exhaustive exploration of all options for improving the parameters is beyond the scope of this article.

image1overf

This method performs well for all scenes tested, including preserving the overall flux levels without oversubtracting "background". The individual amplifier option works well on the sparse fields but introduces slight artifacts in the full fields. The code can also be run without source masking; the results are nearly identical to those with it. The background fitting and median cleaning algorithm used in clean_flicker_noise  are based on the image1overf algorithm.

General description and assumptions

This method performs a removal of 1/f readout noise on NIRISS or NIRCam level 2 calibrated image by masking objects and subtracting the median of each column. The correction method assumes that there isn't a significant variation in the background across the field. The code has been designed to handle 1/f stripes either assuming a constant noise along the full detector rows, or amplifier-to-amplifier 1/f noise differences. Detector orientation is determined automatically by searching for the value of the header keyword SLOWAXIS. Only two subarrays are supported: FULL (NIRCam and NIRISS) and SUB256 (NIRISS) subarrays. It includes the effect of a variable background (by temporarily removing it before determining the 1/f correction and adding it back afterwards) and masking pixels containing sources. The author emphasizes that the code should be used with care and the users need to inspect the results for any unintended consequences.

Difference compared to all other packages

By temporarily subtracting a 2-D background to measure and subtract stripes, the pedestal flux level of the image is preserved, which is especially helpful for crowded scenes.

Method steps

1. Subarray type is determined and non-reference pixels are extracted
2. Mask bad pixels, determined from the DQ array
3. Mask all pixels outside ±3-sigma of global median (adjustable with sigma_bgmask parameter)
   1. assumes 1/f noise variations are within this 3-sigma; also assumes background doesn't vary significantly across the field
4. Temporarily subtract 2-D background grid measured using photutils
5. Mask additional pixels one of two ways:
   1. Detect sources using photutils
   2. Mask all pixels outside ±2-sigma of global median of background-subtracted image (adjustable with sigma_1fmask parameter)
6. Measure row stripes: medians of remaining unmasked pixels
   1. Perform within each amplifier individually (ABCD) or across full rows (controlled by splitamps parameter)
   2. Only dominant stripes are corrected, usually rows (sometimes columns) depending on which detector axis is read out slowest (determined from FITS header keyword SLOWAXIS)
7. Subtract row stripes from data (without background subtraction)

remstriping

Originally developed for the CEERS program, this method performs well on the image from that dataset as well as the sparse star field. Since it measures and subtracts the 2-D background, fuller fields are left oversubtracted. Artifacts are introduced when correcting amplifiers individually.

General description and assumptions

Designed for sparse extragalactic "blank" fields observed by programs like CEERS, remstriping measures both row and column striping.

For DR0.5, each 1/f noise-subtracted image was visually inspected and the masking threshold adjusted for the images with over- or under-subtracted pattern noise. For example, images with large, bright sources usually benefit from a threshold closer to 0.1 than 0.8. The threshold can either be changed using the global variable MASKTHRESH, or by setting a value manually for each image.

Originally designed to be run on "rate.fits" files, remstriping includes a step to perform flat fielding before 1/f measurement and correction. The remstriping algorithm can also be run on "cal.fits" files by turning off the flat fielding step.

Method steps

1. Apply flat field (optional, assuming running on "rate" files)
2. Mask sources detected using photutils 1.5 (available option to load source mask)
3. Subtract median (single value, either simple median or mean of Gaussian fit)
4. Measure row and column striping: sigma-clipped median of all unmasked pixels
   1. Measure within each amplifier individually (ABCD) and also across full rows
   2. Use individual amplifier measurements, unless most of the pixels are masked, then revert to the full row/column measurements
   3. Controlled by threshold parameter thresh: default requires 80% of pixels to be good (unmasked and free of detected sources)
5. Subtract stripes from data (including background subtraction, without flat field correction)

ROEBA

Using the default parameters, this method oversubtracts background while leaving some faint striping.

General description and assumptions

ROEBA stands for Row-by-row, Odd/Even By Amplifier correction and is part of the standalone package for reduction of transiting exoplanet data called tshirt. ROEBA has been developed for removal of 1/f noise in the SW channel of NIRCam TSOs (which can differ significantly compared to imaging fields, Schlawin et al. 2021, 2023). Such scenes typically consist of a bright source, whose PSF is defocused using one of the NIRCam SW weak lenses, and usually dominates the scene (in non-crowded fields) with a free of bright sources field providing plenty of background. In order to run ROEBA's correction, the user needs to provide a background region to employ as a proxy for reference pixels that have minimal contamination from bright sources. The package relies on the assumption for a clean background region and could potentially over-subtract light or background sources where this condition is not met.

Difference compared to all other packages

Requires user-provided background mask. Corrects for odd/even effect and uses row medians from each amplifier, or full row to correct for 1/f noise.

Method steps

1. By employing a user-provided background image, ROEBA applies correction in the slow-read-out direction, which subtracts the median odd count level from all odd-numbered columns and the median count level from all even-numbered columns; this first step removes the pre-amplifier offsets;
2. Calculates row-by-row median levels from the background region and subtracts this from all pixels in the given row; this reduces the 1/f noise
   1. controlled by amplifiers parameter

NSClean

This method was originally developed for NIRSpec MSA observations including many blank, unilluminated pixels to precisely measure and subtract background Fourier modes, using a robust theoretical basis. It can also be applied to NIRCam images of individual objects (coronagraphy or TSO) surrounded by many "blank" pixels. It was not designed for the scenes tested above, containing many objects. In those cases, additional Fourier modes are detected, measured, and subtracted, leaving artifacts, even in "blank" fields. 

This method is implemented in the clean_flicker_noise step for NIRSpec in version 1.16.0 of the JWST Science Calibration pipeline.

smooth1overf

Designed to simply smooth the row and column medians, this method performs well in the general case, while leaving some residual striping, especially on larger scales (more than a few rows).

General description and assumptions

This simple method calculates sigma-clipped medians of each row / column, then smooths them with simple rolling averages. Shown below are the original (left) and corrected (right) images along with the sigma-clipped medians of columns (top) and rows (right) in blue, and the smoothed medians in orange used to correct the images.

destriper

Another simple method, this median smoothing works well on sparse fields. By processing amplifiers individually, artifacts are introduced in more full scenes.

General description and assumptions

The destriper package takes as input calibrated or "rate" NIRCam images, and corrects for 1/f noise per amplifier by removing the median level of each row. There is no assumption for the user to provide background regions for determining the median for each row.

Difference compared to all other packages

Relies on median level of amplifier image rows without source masking.  

Method steps (per individual NIRCam amplifier)

1. Extract image area without the reference pixels from the science extension
2. Compute median level for each row
3. Subtract the median for each row and offset to the median of the input data
4. Subtract median for each amplifier and offset to the median level of the first amplifier

---

References

Schlawin, E., et al. 2021, AJ, 161, 115
JWST Noise Floor. II. Systematic Error Sources in JWST NIRCam Time Series

Schlawin, E., et al. 2023, PASP, 135, 8001
JWST NIRCam Defocused Imaging: Photometric Stability Performance and How It Can Sense Mirror Tilts

---