Algorithms for calwebb_detector1, which is stage 1 of the JWST Science Calibration Pipeline, are used to process raw ramps data into uncalibrated slopes data. 

The calwebb_detector1 module is stage 1 of the JWST Science Calibration Pipeline. It processes data taken with all instruments and modes. The input to this stage is the raw non-destructively read ramps and the output is uncalibrated slope images. A number of detector-level corrections are performed in this stage and slopes are fit to the corrected ramps. The steps are listed in Figure 1 with the flow from the top to the bottom.

It is important to note that these initial steps are applied to each detector individually (so, for example, NIRCam images that are obtained with multiple detectors simultaneously will go through these initial calibration steps using calibration reference files that are specific to each individual detector, such as superbias, dark current subtraction, and also other detector-specific reference files such as flat fields, photometric calibration, distortion, etc., when those are applied in later steps in the pipeline).

Unless otherwise stated, the algorithms described are the baseline version.

---

Steps for both NIR and MIR data

Data quality initialization

Software documentation outside JDox: Data Quality (DQ) Initialization, Error Propagation 

Data quality flags on the pixels are initialized. The pixel data quality flags are initialized based on detector-specific calibration reference files and designate permanent conditions that affect all groups for that pixel. Examples of such flags are dead pixels, hot pixels (showing excess dark signal), and low quantum efficiency pixels. The group data quality flags are initialized to zero, and these flags would be set if conditions occur that only affect some groups for a pixel. Both the pixel and group flags are updated during subsequent calibration pipeline steps as needed.

This step also initializes the uncertainties for each pixel (error arrays). Their values are modified during subsequent steps, and are propagated in the calibration pipeline using a noise model. The uncertainty from each step that contributes noise to the final measurement is separately propagated through the science calibration pipeline. Different uncertainty sources behave in different ways. Some noise sources (e.g., photon noise) are independent between integrations and others (e.g., flat field noise) are not. In addition, the spatial covariance of different sources varies. By propagating each term through the calibration pipeline, the use of each term can be customized for the processing. For example, the use of the flat field noise term is different between non-dithered and dithered observations. For the former, the noise does not reduce with the addition of more integrations while for the latter it does.

Saturation check

Software documentation outside JDox: Saturation Detection 

The analog-to-digital (ADU) values for each group are checked for saturation. The saturation level is set by detector-specific calibration reference files. The group data quality flag is set if the DN value in the group is above the reference file set saturation level. 

The baseline version of this algorithm does not account for the case where not all of the frames in a group are saturated (some are and some are not). An optimal version of this algorithm that does account for this case is currently under investigation.

Reference pixel correction

Software documentation outside JDox: Reference Pixel Correction 

All the detectors have reference pixels that have the same readout electronics as regular pixels, but are not sensitive to light. These pixels are located at the edge of the detector arrays and are read out by the same amplifiers as the regular pixels. Thus, reference pixels track drifts in the readout electronics. The average reference pixel level for each readout amplifier is subtracted from the regular pixels for that same amplifier.

Jump detection

Software documentation outside JDox: Jump Detection 

All the detectors can show ramp jumps where the ADU level between 2 consecutive groups is large relative to those between other consecutive pairs of groups. These ramp jumps are most often caused by cosmic rays that instantaneously deposit large amounts of charge in a pixel. Ramp jumps can be detected and flagged. The number of sigmas above the noise threshold is given as a parameter.

For the baseline algorithm, the 2-point difference method is used as it is computationally fast and sufficient for measurements in the photon dominated regime (see Anderson & Gordon, 2011). An optimal version of this algorithm that detects smaller ramp jumps in the read noise regime is under investigation.

The jump detection step has recently been enhanced (as of pipeline version 1.8.0) to detect and flag "snowballs and showers" that result from unusually energetic cosmic ray events which produce halos around the central pixels. This step will be enabled for automatic processing in the pipeline in future, and for the moment it can be enabled manually when running the pipeline offline, with the ability to set parameters that describe the criteria and thresholds for flagging these snowballs. More details are available on the pipeline software documentation pages for the jump detection step (Jump Detection).

It should be noted that if a moving object is present in observations that are specified as fixed target, this step can can erase the signal from such an object (and vice versa; for observations that are specified as moving target, this step can erase the signal from fixed objects such as stars or galaxies which will trail across the detector during the integrations). This is discussed further in this article: .Moving Target Calibration and Processing.

Slope fitting

Software documentation outside JDox: Ramp Fitting 

The slope for each ramp is determined by performing a weighted linear least squares fit. If a ramp has flagged ramp jumps, the fit is done for each jump-free segment and the resulting slopes averaged to produce a single slope per ramp. The weighting is done using the Fixsen et al. (2000) "optimal weighting" method.

An enhanced version of this algorithm that uses generalized least squares and a covariance matrix is under investigation.

---

NIR-specific steps

Group scale correction

Software documentation outside JDox: Group Scale Correction 

In rare cases, groups can be taken where the number of frames in a group is not a power of 2. This results in the onboard software not dividing by the appropriate number of frames, as the onboard software does averaging by simple bit-shifting. The information on the number of actual frames in a group and the onboard assumed number of frames in the group is known. This step multiplies all the raw DN values by the appropriate ratio to provide the correct DN values for the actual number of frames averaged per group.

Superbias subtraction

Software documentation outside JDox: Superbias Subtraction 

The overall DN level in each pixel and group is offset by a bias level. The nonlinearity in a ramp is relative to this bias level. Thus, the bias level in each group for each pixel is subtracted based on detector-specific calibration reference files to provide the correct DN level for the nonlinearity correction step.

Linearity correction

Software documentation outside JDox: Linearity Correction 

The NIR detectors show a nonlinearity that is due to the gain changing. This non-linearity is well fit with a low order polynomial fit versus relative DN where relative is referenced to the bias level. The correction is done using the fitted polynomials whose parameters are provided by detector-specific calibration reference files.

For the baseline algorithm, grouped data is treated the same as for non-grouped data, and the error due to this assumption is small. An enhanced version of this algorithm that accounts for the effects of grouping on the nonlinearity correction is under investigation.

Persistence correction

Software documentation outside JDox: Persistence Correction 

The detectors suffer from persistence giving rise to faint "after images" of previous exposures that are seen in the current exposure. The persistence decays exponentially. The persistence is corrected using a trap based model of the persistence with the model details given by detector-specific calibration reference files. In addition to correcting the persistence, a pixel flag is set indicating that persistence was corrected at a detectable level.

Dark subtraction

Software documentation outside JDox: Dark Current Subtraction 

The NIR detectors show excess signal in dark exposures. This excess signal is subtracted group-by-group using detector-specific calibration reference files.

Gain scale correction

Software documentation outside JDox: Gain Scale Correction 

In the case of subarray observations taken with a different gain than full array observations, the non-standard gain means that the measured DN values need to be corrected to put them into the same gain reference as full array observations, to enable all downstream processing to work seamlessly. This is done by scaling the slope and slope uncertainties appropriately using the ratio of the non-standard and standard gains.

---

MIR-specific steps

Reset anomaly correction

Software documentation outside JDox: Reset Anomaly Correction 

The MIR detectors show a transient phenomenon at the beginnings of ramps that is due to the reset. This transient is additive, and is caused by the non-ideal behavior of the field effect transistor (FET) upon resetting in the dark, causing the initial frames in an integration to be offset from their expected values. The first 12 groups in MIR ramps should have the reset anomaly subtracted. This correction is derived from dark observations and behaves similarly to the dark subtraction step. This correction is integration dependent.

For the baseline algorithm, the reset anomaly is separated from the dark subtraction step to simplify other MIR correction steps in the calibration pipeline.

First frame correction

Software documentation outside JDox: First Frame Correction 

The 1st frame of MIR data has a transient that has not been fully characterized.

For the baseline algorithm, this 1st frame is flagged and not used further in the calibration pipeline. An enhanced version of this algorithm that corrects for this transient is under investigation.

Last frame correction

Software documentation outside JDox: Last Frame Correction 

The MIR detectors show a transient in the last frame that is caused by the reset pattern. The last frame of the MIR detectors is a read-reset frame where 2 rows are read and then reset. Then, the next 2 rows are processed in the same manner. The transient is caused by the reset strongly changing the values in the to-be-read pixels due to the coupling between pixels. 

For the baseline algorithm, the last frame is flagged and not used further in the calibration pipeline.

An enhanced version of this algorithm is under development. The amplitude of the transient seems to be well characterized by a polynomial fit to the value in the last frame itself. The correction under development will use polynomial parameters provided in detector-specific calibration reference files.

Linearity correction

Software documentation outside JDox: Linearity Correction 

The MIR detectors show a non-linearity that is due to changing quantum efficiency. This non-linearity is well fit with a low order polynomial fit versus absolute DN. The non-linearity is wavelength-dependent at wavelengths above approximately 21 μm. The correction is done using the fitted polynomials whose parameters are provided by detector and wavelength-specific calibration reference files.

RSCD correction

Software documentation outside JDox: Reset Switch Charge Decay (RSCD) Correction 

The "reset switch charge decay" is a transient seen in the 2nd and subsequent integrations in a MIR exposure. If uncorrected, this transient results in the slopes in the 2nd and higher integrations being larger than the 1st integration. This transient is well described by a decaying single exponential that is proportional to the counts at the end of the previous integration. The correction involves subtracting this exponential from the 2nd and higher integrations in multi-integration ramps where the parameters of the exponential are set by detector-specific calibration reference files.

Dark subtraction

Software documentation outside JDox: Dark Current Subtraction 

The MIR darks show a dependence on the integration number in an exposure. The dark subtraction is done group-by-group using detector and integration specific calibration reference files.

---

References

Anderson & Gordon 2011, PASP, 123, 1237
Optimal Cosmic-Ray Detection for Nondestructive Read Ramps

Fixsen et al. 2000, PASP, 112, 1350
Cosmic-Ray Rejection and Readout Efficiency for Large-Area Arrays

---

---