NIRSpec BOTS Known Issues
Known issues specific to NIRSpec BOTS (bright object time series) spectroscopic data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. Users should also refer to the Known Issues with JWST Data page for characteristics and caveats that are common to all instruments in time-series (TSO) mode.
On this page
Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.
Please also refer to NIRSpec BOTS Calibration Status for an overview of the current photometric and wavelength calibration accuracy of NIRSpec BOTS data products.
Artifacts
Information on NIRSpec instrument artifacts are found on the main NIRSpec Known Issues article.
Pipeline notes
Early in the mission, our understanding of the observatory performance is evolving quite rapidly and changes to calibration procedures are expected. Spectrophotometry obtained using NIRSpec/BOTS during commissioning revealed that the instrument performance is close to photon and read-noise limited, in part due to the excellent stability of the observatory (Espinoza et al., 2022).
However, at this moment, there are some caveats in the pipeline processing that observers should be aware of that are summarized below.
Stage 1 processing
1/f noise
See also: JWST Time-Series Observations Noise Sources
The NIRSpec near-infrared HgCdTe detectors show spatially correlated noise, known as 1/f noise, that is introduced by the detector readout system (Schlawin et al., 2020; Moseley et al., 2010). One of the features of this noise is an apparent horizontal striping in the detectors as explained in the NIRSpec Known Issues article and in JWST Time-Series Observations Noise Sources. 1/f noise has a considerable impact on the light curve scatter for time-series observations if it is not accounted for. It can be partially corrected.
One possible way of performing the 1/f correction is by carefully selecting background pixels and subtracting those from each pixel in each column. This methodology is not perfect because it does not account for the intra-column component of 1/f noise nor does it correct for all the inter-column covariance between pixels. One way to take this covariance into account is to include it in the spectral extraction algorithm itself, as suggested by Schlawin et al. (2020, Sections 4.3.3. and 4.3.4). Optimal methods for extracting and/or correcting 2-D spectra considering 1/f noise are under investigation.
The user may consider experimenting with a recently published algorithm for NIRSpec called NSClean by B. Rauscher (2023).
Saturation
See also: JWST Time-Series Observations TSO Saturation
In Birkmann et al. (2022), the authors plot the brightness limits for the available filter and disperser combinations (Figure 1) in J-band Vega magnitudes, calculated using the default subarrays, that is, SUB512 for PRISM/CLEAR and SUB2048 for all gratings. Host stars of known transiting exoplanets, as of September 2021, are over-plotted in gray, with a few labeled by name. If a host star lies beneath the curve for a given configuration, that target can be observed without any saturation in the spectrum. It is clear that most of these targets can be observed with JWST NIRSpec even with the PRISM configuration and only a handful are out of reach for the high-resolution gratings. It is still possible to observe host stars that fall above the curves, but with partial saturation in some wavelength ranges. An uncertainty of 20% should be considered when using brightness limits indicated in Figure 1.
Click on the figure for a larger view.
An uncertainty of 20% should be considered when using brightness limits indicated in this figure. © Birkman et al., 2022
Quantum yield
See also: JWST Time-Series Observations Noise Sources
The HgCdTe detectors employed in NIRSpec are known to occasionally generate more than a single electron-hole pair for incident photons having energies sufficiently above the band gap of the device. Jakobsen (2022) created simple statistical models for the additional noise added to the extracted detector electron signal due to the quantum yield effect. Using pre-flight quantum yield estimates, they demonstrated that photon-noise error bars at 1.5 μm are likely underestimated by 10%–30%; the same effect implies an underestimation of those error bars of 30%–50% at 0.6 μm. The actual magnitude and wavelength dependence of this enhancement is currently under investigation.
Observed TSO systematic effects
The NIRSpec focal plane uses 2 individual detectors, also called sensor chip arrays (SCAs). The SCAs (designated as NRS1 and NRS2) are IR hybrid arrays with HgCdTe used for light detection and a silicon integrated circuit for the readout. More information can be found in the NIRSpec Detectors article.
From a commissioning study performed on HAT-P-14 b (Espinoza et al. 2022) using NIRSpec/BOTS G395H/F290LP, the NRS1 transit light curve clearly shows a more pronounced exposure-long slope than that for the NRS2 (about ~140 ppm per hour for NRS1, compared to 40 ppm per hour for NRS2). While some wavelength-dependence of the slope for NRS1 is observed, the smaller slope seen in the NRS2 light curves is fairly constant for all wavelengths. It is possible that this effect is due to some underlying detector level effect, and this issue is currently under investigation.
Light curve scatter
From the commissioning study performed on HAT-P-14 b (Espinoza et al. 2022) using NIRSpec BOTS G395H/F290LP (3–5 um), at the native resolution of the instrument, the JWST pipeline precisely predicts the noise levels observed on the out-of-transit light curves in the NRS1 detector. However, there appears to be a slight but significant extra scatter of about 5% in the actual data when compared with the pipeline products for NRS2. The source of this discrepancy is currently under investigation.
Summary of common issues and workarounds
The sections above provide details on each of the known issues affecting NIRSpec BOTS data; the table below summarize some of the most likely issues users may encounter along with any workarounds, if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.
| Symptoms | Cause | Workaround | Fix build | Mitigation Plan |
|---|---|---|---|---|
| NS-BOTS02: Large number of outlier pixels are identified in the jump step, which leads to lower SNR of the "rateints" products (slope images) per integration. | The jump step default threshold, in combination with the current reference file uncertainties, results in too many pixels being flagged as cosmic rays. | Rerun the jump step with the a rejection_threshold value > 10 (default is 4.0). | N/A | Updated issue None at this time. Updates await investigation into optimal jump thresholds. |
| NS-BOTS03: There is vertical striping across the NIRSpec BOTS rate images (2-D spectra), which appears as negative and/or surplus flux in extracted 1-D spectra (typically with an irregular wavelength-dependent undulation). | Correlated noise from low level detector thermal instabilities (1/f noise) is seen as vertical banding in 2-D count rate images, particularly in exposures of the NRS2 detector. | Run NSClean script developed by B. Rauscher on count rate images, using an appropriate mask. Several additional community tools that mitigate 1/f noise are also available. | N/A | Updated issue NSClean was implemented in the pipeline (jwst 1.13.0 and onwards) as a non-default option. Details on how to invoke NSClean within the science calibration pipeline are provided in the workaround notebook. Additional 1/f cleaning options in the pipeline are planned for the future (possibly September 2024), pending further development. |
NS-BOTS04: The extracted 1-D spectrum ("x1d" product) has spurious absorption features, typically one pixel wide. | Pixels marked as "DO_NOT_USE" in the reference data are not included in 1-D extracted spectrum. | Use new science calibration pipeline software (jwst 1.11.0 onwards) to estimate the value for each "DO_NOT_USE" pixel from neighboring 2-D profiles before extracting 1-D spectrum. This is disabled by default; testing has shown improvements in some cases, but degradation in data quality has also been seen in certain circumstances; users should exercise caution if they choose to run the step. | N/A | Updated issue Test that the pixel_replace algorithm works well for NIRSpec fixed slit data. If it does, enable the pixel_replace step by default in the parameter reference file update (estimated for September 2023). Users can enable this step by following commands in this file. The team is evaluating options for further improvements. |
NS-BOTS05: Artificial ~10% deep absorption feature between about 0.97 and 1.02 µm in NIRSpec G140M/F070LP spectra when observed through the S1600A1 aperture. | The lamp flat field images, which are used to create the "sflat" reference files, for the S1600A1 slit are contaminated by the zero-order image, which causes a large positive feature in the "sflat" correction image in the lower part of the aperture. | Use only the spectra extracted from the upper dither positions in the affected wavelength range. | N/A | Created issue Investigate feasibility of editing the "sflat" reference file to remove the contamination, and/or flagging affected pixels as DO_NOT_USE. |
NS-BOTS06: Fluxes are too low at the red and blue wavelength extremes of extracted 1-D spectra for M- and H-grating observations | The default extract_1d extraction uses a rectangular extraction box that does not follow the curved trace of M- and H-grating data in "calints" files. This causes the 1-D extraction to miss some of the flux at the red and blue wavelength extremes for M-grating data and can miss the red and blue ends of the H-grating traces entirely. | A curved trace can be used for 1-D extraction by providing extract_1d.override_extract1d a modified "extract_1d" parameter reference file that provides a list of polynomial coefficients that define the trace via the src_coeff keyword (see the extract_1d step description for more details). Alternatively see the calwebb_spec2 workaround notebook for BOTS data from JWebbinar 29 for an example of a custom extraction of BOTS data. | N/A | Created issue None at this time. Updates await investigation into potential solutions.
|
| NS-BOTS01: The shape of spectra taken with the SUB512, SUB512S, or SUB32 subarrays exhibits unexpected features and wavelength-dependent flux discrepancies of 10% or more. This causes time-series observations (TSO) light curve scatter to increase by orders of magnitude larger than expected. | Some subarrays have no reference pixels, which means bias drifts are not corrected. | Rerun calwebb_detector 1 starting from the "_uncal" files. Before the linearity step, identify non-iluminated pixels in the detector and estimate their median value; this will provide an estimate of the detector pedestal level on each group. Remove this value from each group, and then run the remaining calwebb_detector1 steps. | Updated Operations Pipeline The reference pixel correction was modified to use unilluminated pixels in the columns at the left and right edges of these subarrays. This was implemented in the Operations Pipeline on August 24, 2023. |
References
Birkmann, S. M., et al., 2022, A&A, 661A, 83B
The Near-Infrared Spectrograph (NIRSpec) on the James Webb Space Telescope. IV. Capabilities and predicted performance for exoplanet characterization
Espinoza, N., et al. 2022, PASP, 135, 018002
Spectroscopic Time-series Performance of JWST/NIRSpec from Commissioning Observations
ADS
Jakobsen, P., 2022, ESA Instrument Report, ESA-JWST-SCI-NRS-TN-2022-003
(PDF)
Moseley, S. H., et al. 2010 SPIE Proceedings Vol. 7742
Reducing the read noise of H2RG detector arrays: eliminating correlated noise with efficient use of reference signals
Rauscher, B.J., 2023, submitted
NSClean: An Algorithm for Removing Correlated Noise from JWST NIRSpec Images
Schlawin, E., et al. 2020, AJ 160, 231S
JWST Noise Floor. I. Random Error Sources in JWST NIRCam Time Series
ADS