NIRSpec Time Series Observations Pipeline Caveats
Unique features of the JWST Science Calibration Pipeline for time-series observations (TSOs) with the NIRSpec instrument, and caveats for users, are described in this article. Users should also refer to the TSO pipeline overview for characteristics and caveats that are common to all instruments. This information reflects the status for the JWST pipeline version 1.6.2.
Summary of specific NIRSpec BOTS pipeline issues
The information in this table about NIRSpec bright object time-series spectroscopy calibration pipeline issues is excerpted from Known Issues with JWST Data Products.
|NS-BOTS02: Large number of outlier pixels are identified in the jump step, which leads to lower SNR of the "rateints" products (slope images) per integration.||The jump step default threshold, in combination with the current reference file uncertainties, results in too many pixels being flagged as cosmic rays.|
Rerun the jump step with the a rejection_threshold value > 10 (default is 4.0).
A mitigation plan is under development.
|NS-BOTS03: There is vertical striping across the NIRSpec BOTS rate images (2-D spectra), which appears as negative and/or surplus flux in extracted 1-D spectra (typically with an irregular wavelength-dependent undulation).||Correlated noise from low level detector thermal instabilities (1/f noise) is seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector.|
Eventual inclusion of a cleaning algorithm in the pipeline is planned, pending further testing, possibly in February 2024.
NS-BOTS04: The extracted 1-D spectrum ("x1d" product) has spurious absorption features, typically one pixel wide.
|Pixels marked as "DO_NOT_USE" in the reference data are not included in 1-D extracted spectrum.|
Use new science calibration pipeline software (jwst 1.11.0 onwards) to estimate the value for each "DO_NOT_USE" pixel from neighboring 2-D profiles before extracting 1-D spectrum. This is disabled by default, and has not yet been tested; systematic errors may be introduced in TSO data.
Another workaround would be to remove "DO_NOT_USE" DQ flags on affected pixels. See this demo on how to do it, based on NIRISS SOSS data.
Test that the pixel_replace algorithm works well for NIRSpec fixed slit data. If it does, enable the pixel_replace step by default in the parameter reference file update (estimated for September 2023). Users can enable this step by following commands in this file.
NS-BOTS05: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture.
The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture.
Use only the spectra extracted from the upper dither positions in the affected wavelength range.
Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE.
|NS-BOTS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. This causes time-series observations (TSO) light curve scatter to increase by orders of magnitude larger than expected.|
Some subarrays have no reference pixels, which means bias drifts are not corrected.
Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps.
Updated Operations Pipeline
The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. This was implemented in the Operations Pipeline on August 24, 2023.
About NIRSpec TSO pipeline caveats
Early in the mission, our understanding of the observatory performance is evolving quite rapidly and changes to calibration procedures are expected. Spectrophotometry obtained using NIRSpec/BOTS during commissioning revealed that the instrument performance is close to photon and read-noise limited, in part due to the excellent stability of the observatory (Espinoza et al., 2022).
However, at this moment, there are some caveats in the pipeline processing that observers should be aware of that are summarized below.
Stage 1 processing
The NIRSpec near-infrared HgCdTe detectors show spatially correlated noise, known as 1/f noise, that is introduced by the detector readout system (Schlawin et al., 2020; Moseley et al., 2010). One of the features of this noise is an apparent horizontal striping in the detectors as explained in the NIRSpec Instrument Features and Caveats article and in JWST Time-Series Observations Noise Sources. 1/f noise has a considerable impact on the light curve scatter for time-series observations if it is not accounted for. It can be partially corrected.
One possible way of performing the 1/f correction is by carefully selecting background pixels and subtracting those from each pixel in each column. This methodology is not perfect because it does not account for the intra-column component of 1/f noise nor does it correct for all the inter-column covariance between pixels. One way to take this covariance into account is to include it in the spectral extraction algorithm itself, as suggested by Schlawin et al. (2020, Sections 4.3.3. and 4.3.4). Optimal methods for extracting and/or correcting 2-D spectra considering 1/f noise are under investigation.
The user may consider experimenting with a recently published algorithm for NIRSpec called NSClean by B. Rauscher (2023).
In Birkmann et al. (2022), the authors plot the brightness limits for the available filter and disperser combinations (Figure 1) in J-band Vega magnitudes, calculated using the default subarrays, that is, SUB512 for PRISM/CLEAR and SUB2048 for all gratings. Host stars of known transiting exoplanets, as of September 2021, are over-plotted in gray, with a few labeled by name. If a host star lies beneath the curve for a given configuration, that target can be observed without any saturation in the spectrum. It is clear that most of these targets can be observed with JWST NIRSpec even with the PRISM configuration and only a handful are out of reach for the high-resolution gratings. It is still possible to observe host stars that fall above the curves, but with partial saturation in some wavelength ranges. An uncertainty of 20% should be considered when using brightness limits indicated in Figure 1.
The HgCdTe detectors employed in NIRSpec are known to occasionally generate more than a single electron-hole pair for incident photons having energies sufficiently above the band gap of the device. Jakobsen (2022) and Espinoza & Volk (2022) created simple statistical models for the additional noise added to the extracted detector electron signal due to the quantum yield effect. Using pre-flight quantum yield estimates, they demonstrated that photon-noise error bars at 1.5 μm are likely underestimated by 10%–30%; the same effect implies an underestimation of those error bars of 30%–50% at 0.6 μm. The actual magnitude and wavelength dependence of this enhancement is currently under investigation.
Observed TSO systematic effects
The NIRSpec focal plane uses 2 individual detectors, also called sensor chip arrays (SCAs). The SCAs (designated as NRS1 and NRS2) are IR hybrid arrays with HgCdTe used for light detection and a silicon integrated circuit for the readout. More information can be found in the NIRSpec Detectors article.
From a commissioning study performed on HAT-P-14 b (Espinoza et al. 2022) using NIRSpec/BOTS G395H/F290LP, the NRS1 transit light curve clearly shows a more pronounced exposure-long slope than that for the NRS2 (about ~140 ppm per hour for NRS1, compared to 40 ppm per hour for NRS2). While some wavelength-dependence of the slope for NRS1 is observed, the smaller slope seen in the NRS2 light curves is fairly constant for all wavelengths. It is possible that this effect is due to some underlying detector level effect, and this issue is currently under investigation.
Light curve scatter
From the commissioning study performed on HAT-P-14 b (Espinoza et al. 2022) using NIRSpec BOTS G395H/F290LP (3–5 um), at the native resolution of the instrument, the JWST pipeline precisely predicts the noise levels observed on the out-of-transit light curves in the NRS1 detector. However, there appears to be a slight but significant extra scatter of about 5% in the actual data when compared with the pipeline products for NRS2. The source of this discrepancy is currently under investigation.
Birkmann, S. M., et al., 2022, A&A, 661A, 83B
The Near-Infrared Spectrograph (NIRSpec) on the James Webb Space Telescope. IV. Capabilities and predicted performance for exoplanet characterization
Espinoza, N., et al. 2022, PASP, in press
Spectroscopic Time-series Performance of JWST/NIRSpec from Commissioning Observations
Espinoza & Volk, 2022, STScI Report (under review)
Jakobsen, P., 2022, ESA Instrument Report, ESA-JWST-SCI-NRS-TN-2022-003
Moseley, S. H., et al. 2010 SPIE Proceedings Vol. 7742
Reducing the read noise of H2RG detector arrays: eliminating correlated noise with efficient use of reference signals
Rauscher, B.J., 2023, submitted
NSClean: An Algorithm for Removing Correlated Noise from JWST NIRSpec Images