Known issues specific to NIRSpec MOS data processing in the JWST Science Calibration Pipeline are described in this article. This is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of issues that users should be aware of for their science. 

Specific artifacts are described in the Artifacts section below. Guidance on using the pipeline data products is provided in the Pipeline Notes section along with a summary of some common issues and workarounds in the summary section.

Please also refer to NIRSpec MOS Calibration Status for an overview of the current astrometric, photometric, and wavelength calibration accuracy of NIRSpec MOS data products.

Artifacts

Information on NIRSpec instrument artifacts are found on the main NIRSpec Known Issues page.

Pipeline notes

Description of major issues

Correlated 1/f read noise

See NIRSpec Known Issues for more details.

The effects of 1/f noise for NIRSpec/MOS are shown in MOS 1/f noise workaround notebook, which also demonstrates the use of the NSClean algorithm to remove most of this effect. NSClean is now implemented in the pipeline (v1.13.4 onwards) as a non-default option. Further details on how to invoke NSClean within the science calibration pipeline and adjust default parameters will be described in the 1/f noise workaround notebook. 

Pathloss correction

The NIRSpec flux calibration is based on observations of point sources centered in an aperture. Off-centered point sources, the typical case for MOS observations, will have lower counts because more of the PSF will be truncated; the path loss correction is intended to account for the flux lost relative to a centered source. See NIRSpec Calibration Concept for a more detailed description of the path loss correction concept.

For MOS, the pipeline uses the planned source position in the shutter as determined in the MSA planning tool. The actual position will be subject to the MSATA accuracy, which itself depends on the relative astrometric accuracy of the source catalog; in most cases, this should provide reasonable flux accuracy. Users can try to measure the source position directly from TA confirmation images; however, tools or examples showing how that might be accomplished are not yet available.

The current calibration reference files used for the MOS path loss correction were created using a pre-launch PSF model. Because the on-orbit image quality is considerably better, the applied corrections may be somewhat overestimated (particularly at wavelengths shorter than ~ 2 µm). Point source observations will be obtained in spring 2024 to better constrain the calibration.

A bug in the implementation for MOS, which resulted in no correction being applied when a slitlet contains more than 3 open shutters, has been fixed in the latest release build 10.1.

Flux calibration

The flux calibration for MOS data is currently based on observations of a spectrophotometric standard star (the same star for all of the gratings, but a different one for the PRISM) placed near the center of the Q4 longslit. Any field dependence caused by throughput variations in the spectrograph optics or detector should be corrected by the S-flat (described in more detail in the NIRSpec calibration concept page). However, FOV variations in the NIRSpec fore-optics or filter wheel, or the OTE, are not yet taken into account; this is under active investigation using a variety of existing and planned observations. Current best estimates of the flux calibration uncertainties are given on the NIRSpec MOS Calibration Status article.

Wavelength calibration

The wavelength calibration for all modes is based on a parametric instrument model calibrated using internal line lamps. For MOS, a full solution over the field is calculated using multiple open slits distributed over the MSA FOV. The lamps provide uniform illumination of each slit, in which the wavelength scale is equivalent to that of a perfectly-centered point source. Current best estimates of the accuracy of this calibration are given on the NIRSpec MOS Calibration Status article.

Because the vast majority of sources observed with MOS are not centered, a wavelength correction is needed to account for the skewed LSF relative to the centered case. This correction is applied based on the planned source position within its shutter (see the Source centering section below). However, there is a known bug that prevents the correction from propagating to the rectified 2-D spectrum, from which the 1-D spectrum is extracted, so all 1-D spectra of offset point sources do not have the corrected wavelength scale. There can be a discrepancy of up to 1/4 of a resolution element for sources close to the edges of the shutter.

Outlier detection and resampling

The curvature of spectra in the native detector frame can result in artifacts when resampled onto a rectified grid in the "s2d" products. This sometimes resulted in discontinuities in 1-D extracted spectra at nod positions where the source is closer to the edge. This should now be resolved as the size of the 2-D cutouts have been extended well beyond the location of the source trace.

The resampling algorithm has two weighting options, which among other things can effect the detection and masking of outliers. One option (ivm) is based on the read noise per pixel, while the other (exptime) simply weights by the length of the exposure. The exptime option is now the default, as testing has shown that it produces better results in a majority of cases. However, the ivm option can sometimes show better results for faint read noise-limited sources (see Fig. 1). Users should compare both weighting options if the extracted spectra appear more noisy or show more outliers than expected.

1-D extraction aperture centering

The default centering of the 1-D extraction aperture boxes was originally based on the source coordinates, and was frequently offset from the actual source trace position. This issue has been mitigated by a change to the code that now makes use of the relative position of the source within its shutter, as calculated by the MSA planning tool in APT when the MOS plan was designed. However, small offsets can still occur (for example, from small errors in the target acquisition), so users should manually check the centering and adjust as needed.

Other pipeline and calibration considerations

The automated JWST pipeline for MOS data processing has been designed around "typical" use cases (e.g., multiple slitlets consisting of one or more open shutters, each containing a single source, with or without nodding). Given the range of configurations possible for the MSA, and many different types of sources, not every observation can be treated optimally by default. Outlined below are several calibration considerations specific to MOS observing, known limitations of the current state of the pipeline, and some aspects that can or should be changed through reprocessing of the data.

Background subtraction: nods vs master background

There are 2 processing flows for background subtraction of NIRSpec data: nod or master background subtraction. In the MOS case, nod subtraction is done automatically when the observation contains 2 or more associated exposures in which the sources have been moved to different shutters in their slitlets for the same MSA configuration. This is a straight subtraction of full frame images, performed before the 2-D spectrum of each source is extracted. For this reason, if nodded background subtraction is desired for a subset of sources on a configuration, the data will have to be processed twice: once with nodding, and again with master background subtraction. The efficacy of this approach depends primarily on the spatial extent of the sources vs. the separation of the nods.

In the master background approach, dedicated background slitlets are opened on areas in the field of view that ideally do not contain sources. The pipeline automatically extracts and combines these into a master 1-D background spectrum, which is then resampled into the 2-D space of each source spectrum and subtracted. This method improves observing efficiency, and is preferred for significantly extended sources; its efficacy depends on the S/N of the master background, the level of uniformity of the background over the field, and the resampling accuracy. The set of slitlets (and particular shutters within slitlets) used to construct the master background can be changed, for example to remove one that contains a previously unknown contaminating source, by editing the MSA metafile and reprocessing. Note that the failed open shutters and fixed slits can also be used to help construct the master background; however, the workflow is more complicated. Users who wish to explore these options should contact the JWST Help Desk for instructions.

Source type determination

The processing flow and some of the throughput corrections that are applied are different depending on whether a source is point or extended. For MOS data, the pipeline determines the source type for each source based on the value of the stellarity parameter in the MSA metafile. For stellarity > 0.75, the source type is set to "POINT"; for stellarity between 0 and 0.75, source type is set to "EXTENDED". The stellarity values are propagated from the source catalog used in MPT; if the catalog did not specify stellarity, then the pipeline defaults to a source type "POINT".

Note that for NIRCam pre-imaging or any other JWST imaging observations, the source catalog generated by the imaging pipeline does not calculate stellarity. Instead, a flag called "IS_EXTENDED" is set for each source depending on its measured concentration indices. In these cases, MPT converts this flag to the corresponding stellarity limits (e.g., IS_EXTENDED = 0 becomes stellarity = 1), and passes those values to the Proposal Planning Subsystem (PPS) database where they are eventually incorporated into the metafile(s).

The source type assignment by the pipeline can be overridden for any given source by changing the stellarity value in the MSA metafile to give the desired result.

Source centering

For point source processing, several steps depend on the relative position of the source within its shutter (path loss correction, wavelength zero point correction for offset point sources). The automated pipeline uses the positions as planned in MPT, stored in the "ESTIMATED_SOURCE_IN_SHUTTER_X/Y" columns in the MSA metafile. These positions may be slightly different from the actual values in an exposure, depending on the accuracy of the MSATA. If a confirmation image was taken, the actual position of each source can be measured, converted to relative slit units, the values in the MSA metafile replaced, and the data then reprocessed to improve the calibration accuracy (however, any improvement would depend on the measurement accuracy of the source centroid). A future enhancement is planned to automate such a process in the pipeline, but for now this can only be done manually.

Calibration of extended sources

Two source type-dependent processing flows have been designed for 2 limiting cases: spatially unresolved point sources and uniform illumination of a slit aperture. Most, if not all, extended sources will lie somewhere in between these 2 limits. The calibration of such sources is dependent on the exact source geometry as a function of wavelength, and is thus unique to each source and cannot be fully treated by any automated pipeline. Users will need to customize the reprocessing of the data in these cases, in particular a correction for the geometric slit losses. In commissioning, the 2-D slit loss function is characterized by observing a point source at a grid of positions over the open area of a microshutter, for a small subset of shutters. This can then be convolved by the known source geometry (e.g., through forward modeling) to estimate the total slit losses for the integrated spectrum.

Radiometric calibration chain

The MOS flux calibration is set by various throughput corrections and unit conversions applied in the flat field step of the pipeline, in addition to other related corrections such as path loss. NIRSpec Calibration Concept article includes a more detailed description of the various steps in the radiometric correction chain used for all modes, with some information specific to MOS.

1-D extraction 

The size of the 1-D extraction aperture in the cross-dispersion direction is defined by the reference file for the extract_1d step of the calwebb_spec2 pipeline, via the parameter extract_width (in units of pixels). For MOS observations, when the source type is a point source, the center of the extraction aperture is automatically offset to the expected position of the source within its slitlet. The automatic centering can be overridden by setting the optional step argument use_source_posn to False, and modifying the reference file parameters ystart and ystop to define the desired boundary of the extraction aperture (in integer pixels or the use of polynomial  for fractional pixels). The extract_1d step parameters are global, and currently cannot be modified for different sources in the same exposure. There are other optional reference file parameters that allow specification of a boxcar extraction with height that varies with wavelength (the default is a simple rectangle), as well as a background region for background subtraction at the 1-D extraction stage; these options have not been adequately tested and are not recommended at present. Note that aperture corrections for extraction apertures that are larger or smaller than the default width are not yet available, so in such cases users must calculate their own corrections using point source data. Because of how the flux calibration is currently determined, using a different extraction aperture width without applying a correction will result in incorrect absolute flux values.

Unidentified optical shorts

One known characteristic of the MSA is that electrical shorts can occur at a particular shutter location. There is a procedure in place that identifies these shorts using telemetry of electrical currents and masks them in order to prevent the possibility of damage to the array. However, some shorts are too weak to be readily identified via elevated current, but rather manifest themselves by producing a glow caused by heating of the area around the shutter. These "optical" shorts are identified using dark exposures and subsequently masked; while the incidence of new shorts is expected to be very low, there is a possibility that some may appear in between calibration checks during normal operations, which could then contaminate science data. This would manifest in dispersed data as a spectrum with roughly a blackbody shape. It's recommended that users check for any unexpected dispersed signal in their images not related to the already-known failed open shutters (which are flagged in the DQ array of the level 2b data products). Nod subtraction should remove this signal, though the noise level will be somewhat elevated.

Intermittent optical short

An intermittent optical short has been identified on detector NRS1 at the position of (x,y) = (1975, 676) in non-dispersed images such as MOS confirmation images. In dispersed light the short is identified as a bright line of length typical of bright sources for the particular grating/filter combination of the observation. While bright optical shorts are masked (removal of rows and columns that define the short location) a faint intermittent optical short may not be masked if its being masked would remove more pixels than would typically impact one or two sources. The optical short shown in Figure 3 is intermittent and its occurrence has not been tied to particular instrument setups. Individual images and spectra need to be reviewed to determine its presence. 

Long slit processing

Observations using one of the pre-defined long slit configurations (Q4 field point 1 or 2) require somewhat different handling compared to the slitlet case. Because the long slits span the full breadth of 2 MSA quadrants, containing a total of 342 shutters (including some non-operable and vignetted), each shutter needs to be extracted separately in order to preserve wavelength and positional accuracy. This is contrary to the slitlet case, where the calibration is based on the single shutter that contains the source in a given exposure. The full processing flow for the long slit case, particularly in terms of how the metadata should be generated by MPT, is still being defined. For now, users should reprocess long slit data through the calwebb_spec2 pipeline, starting with the level 2a (rate.fits) product, with a modified MSA metafile as described below:

• each shutter in the long slit should be specified as a separate "slitlet" with unique slitlet ID in the SHUTTER_INFO table
• each shutter that is expected to contain light from the target source should be given a placeholder source in the SHUTTER_INFO table, with unique SOURCE_ID, BACKGROUND = N, PRIMARY_SOURCE = Y, DITHER_POINT_INDEX = 1
• each source ID set above should have a corresponding entry in the SOURCE_INFO table, with arbitrary coordinates and stellarity = 0 (to ensure that the extended source processing flow is used)
• each shutter that is expected to contain only background light should be designated a background shutter in the SHUTTER_INFO table (BACKGROUND = Y, PRIMARY_SOURCE = N); these will then be used to generate a master background (if background subtraction is not desired, this can be skipped by specifying sources in all shutters) 

The primary data product will then be a calibrated 2-D spectrum for each designated source shutter. In cases where the long slit has been "stepped" across the source in the dispersion direction, the 2-D spectra can in principle be combined to form a data cube analogous to that of the IFU case. However, the pipeline currently has no mechanism for building such a cube from MOS data—adapting the IFU cube_build step for this purpose is a future planned enhancement. Also note that the caveats given above for extended source calibration apply here, so combination of overlapping dithered exposures should be done with extreme caution.

Level 3 spectral combination

Users should treat automated level 3 products with caution. Some important considerations depending on the source type and specific science requirements:

• Pipeline stage 3 processing includes an outlier rejection step that depends on the noise properties of the individual exposure-level input data. The "crf.fits" output of this step should be checked to verify the performance. If too many pixels are being rejected, raise the S/N threshold using the snr step argument until the results look reasonable. The step should be turned off if the number of input exposures is small (~3 or less).
  
  
• As mentioned above, accurate combined products for extended sources require custom slit loss corrections that the automated pipeline cannot treat.
  
  
• For observations that include widely-separated dithers in which each source is located at very different relative slit locations, the spatial profile of a point source will vary considerably because of the different slit losses. The pipeline does not apply a correction to the spatial profile, so the combination of 2-D data may lead to larger uncertainties in the combined "s2d" product. For such cases, users may want to combine the exposure-level "x1d" products instead by running the optional combine_1d step in place of the resample_spec and extract_1d steps of the calwebb_spec3 pipeline.

Summary of common issues and workarounds

The sections above provide detail on each of the known issues affecting NIRSpec MOS data; the table below summarizes some of the most likely issues users may encounter along with any workarounds if available. Note that greyed-out issues have been retired, and are fixed as of the indicated pipeline build.