Features and potential caveats specific to MIRI MRS data processing in the JWST Science Calibration Pipeline are described in this article.

An example Jupyter notebook that users can use to reprocess data with the pipeline is available on GitHub. 

MIRI MRS pipeline

Astrometric calibration

The astrometric alignment of the 12 MRS bands is generally good to better than 0.1". However, there are some uncorrected higher-order scale effects and linear offsets peculiar to individual slices that are not presently accounted for in the spatial distortion reference files used to assign world coordinates to the dispersed MRS spectral data. This is expected to have little effect on the majority of science programs, other than representing a slight broadening to the reconstructed point spread function. However, the suboptimal alignment can limit the performance and repeatability of the MRS for extremely high SNR studies or those that rely critically on measuring the size and shape of the point spread function. These calibrations will be improved in Cycle 1. 

• Pointing accuracy is generally ~0.3''. However, the WCS will only be as good as the guide star used. Therefore, the WCS of the data may only be accurate to within 1.0" due to inaccuracies in the guide star catalog.
• Simultaneous imaging can be used to refine the WCS based on stars within the FOV. This is not yet part of the main pipeline.

Wavelength calibration

The original commissioning wavelength solution (distortion version FLT-2) had wavelength errors of up to 0.04 μm depending on the channel, as well as an offset in some slices around 17.4 μm resulting in splitting of emission lines crossing these slices. This solution was improved significantly in Cycle1, with the distortion version FLT-4 generally accurate to about 10–20 km/s throughout the MRS wavelength range (Figure 1). FLT-4 was implemented in CRDS OPS context "jwst_0970.pmap" (2022-09-12) of the calibration reference files.  There are some known exceptions to the FLT-4 wavelength accuracy though, with some slices and/or sections of wavelength incorrect by as much as 50 km/s far from the spectral lines used to derive the calibration. Work is ongoing to further improve the wavelength solution and address these remaining issues.

Note that there are 2 ways that the wavelength solution can be given for MIRI MRS data cubes and extracted spectra. Per-band or per-channel cubes are constructed using a linear wavelength solution, the parameters of which are given using the usual FITS CRVAL, CDELT, etc., header keywords. FITS headers use a 1-based counting convention while Python uses a 0-based convention, so the wavelength solution using NumPy for a given data cube would be:

wave = (np.arange(hdr['NAXIS3']) + hdr['CRPIX3'] - 1) * hdr['CDELT3'] + hdr['CRVAL3']

Multi-channel cubes use a nonlinear wavelength solution that cannot be described by FITS header keywords alone. These cubes use the WAVE-TAB convention in which the wavelength coordinates are stored in a FITS table extension.

Note that the wavelength solutions reported by the pipeline have already been corrected for spacecraft motion and are given in the barycentric vacuum rest frame.

Flux calibration

Flux calibration of the MIRI MRS data is presently based on observation of a single standard star during commissioning. As such, the present calibration (particularly longward of about 20 μm) will need to be improved throughout Cycle 1 as more calibration data are obtained. There is also a spectral leak artifact around 12.2 µm which should be corrected before the end of Cycle 1.

Source type

This step determines whether the pipeline should consider the observation to be a point source or an extended source, which affects whether 1-D spectral extraction is later done from the entire FOV or using aperture photometry around a given point.

• If EXTENDED = YES is selected in the APT target description, the pipeline will treat it as an extended source. 
• If EXTENDED = NO in APT, the pipeline will treat it as a point source.  
• If EXTENDED = UNKNOWN, the pipeline will try to use the dither pattern to figure out what the likely source type is. A point source-optimized dither pattern will result in "point", otherwise it will be "extended".  
• This step can be manually overridden to specify either case.
• Time-series observations will require a separate optimal reduction process. For TSO observations the pipeline processing ends before the cube building and spectral extraction steps. (See: JWST Time Series Observations Pipeline Caveats) 

Background subtraction

Software documentation outside JDox: Background Subtraction, Master Background Subtraction

The MIRI MRS sees a significant component of background light at long wavelengths, both from reflected zodiacal light and from JWST thermal emission (particularly in channel 4). As such, it is important to subtract this background signal in order to measure the astrophysical signal of interest. There are presently 3 locations at which this can happen in the pipeline:

1. Background subtraction (calwebb_spec2): This performs pixel-by-pixel background subtraction in which one exposure can be directly subtracted from another, removing a background signal but introducing noise from both the science and background exposures into the final image. This approach is not used by the pipeline unless manually overridden to do so, and analysis of commissioning data shows that it is not necessary in general.
   
   
2. Master background subtraction (calwebb_spec3): This subtracts the background signal from 2-D data using a 1-D master background. Using all x1d products (produced for individual exposures in calwebb_spec2, although they could be created in other ways too) provided in the calwebb_spec3 association file, it obtains the 1-D background spectrum from the "BACKGROUND" extension and projects this spectrum to the entire 2-D detector array. This broadcast array is then subtracted from the individual 2-D science frames with little to no degradation of the SNR (since it uses a 1-D spectral model produced from the sigma-clipped combination of many detector pixels rather than the values of those pixels themselves).
   1. This step does not run by default unless there is a dedicated background observation properly linked to the science observations (Note: There is currently a bug that prevents this step from running at all in the MAST operational pipeline, even when data are linked properly). Science data cubes for programs that did not include dedicated backgrounds will not be background-subtracted unless users create and provide a 1-D background spectrum to the master background routine.
   2. If data are designated as linked background observations, then extract_1d will use the entire field-of-view as the extraction region. The Master Background step recognizes the type of background exposure and uses the appropriate x1d data product to produce the master background. 
      
      
3. extract_1d
   1. (calwebb_spec2): If processing any target indicated as a point source, the extract_1d step will compute a background spectrum using a sigma-clipped algorithm in an annulus surrounding the location of the source. By default, this background spectrum will be subtracted from the 1-D science spectrum extracted from the circular aperture centered on the source (accounting for the effective area of that aperture).
   2. (calwebb_spec3): As in calwebb_spec2, the process of spectral extraction from a 3-D data cube for point sources by default applies a background subtraction from an annular region surrounding a point source. Even in cases for which master background subtraction was already done, the point source aperture correction factor is tied to the expectation that the annular background subtraction has also been performed.

At present, there is a known bug that the pipeline used to produce the spectral data cubes in MAST; it does not apply any of these background subtraction methods, even for data that is correctly linked to dedicated background observations in APT. Note that there was also a bug in the data quality arrays during spectral extraction for version 1.5.3 and earlier of the JWST pipeline that caused the background model produced by the master background step to jump discontinuously between individual MRS bands. This has been fixed in more recent versions.

Outlier detection

Outlier detection does not perform well at present, because the under-sampled nature of the MRS data makes it difficult to find and flag genuine outliers without also flagging unresolved point sources. If users find artifacts in their data, they should try disabling the outlier detection step. Work is ongoing to improve this routine, and its ability to find and flag new bad/hot pixels.

Spectral extraction

Extended sources

For extended sources, spectral extraction sums the flux within the entire MRS field of view. Since the field of view is different for each MRS spectral band (and increases substantially for longer wavelength channels) the summation region will thus be different as well. Depending on the source geometry within the FOV, the extracted 1-D spectrum may thus be discontinuous between bands as it corresponds to a different astrophysical area.

Point sources

For point sources, spectral extraction sums the flux within a conical aperture at the target header coordinates. This conical aperture grows with wavelength to enclose roughly the same total fraction of the PSF, which increases in size by a factor of 5 over the MRS spectral range. Likewise, the annular background subtraction region also grows with wavelength.

Extracted point source spectra for objects with proper motion is currently unreliable. The reported target coordinates in the data product header do not account for any proper motion given in the APT file, even though these are used in the computation of where to point the telescope, and give only the RA/DEC in the epoch provided to APT in the original proposal. Therefore objects with proper motion will have inaccurate header coordinates and the spectral extraction will be done at the wrong location. Sources may therefore need to be re-extracted using adjusted coordinates for targets with proper motion. 

Cube building

Software documentation outside JDox: Cube Building

The cube_build step combines the individual 2-D IFU slice images and creates a 3-D spectral cube. The IFU cubes are by default constructed with north pointing up and east to the left. Cube building can be done using either 3-D drizzle or exponential modified Shepard's method of weighting (emsm), however 3-D drizzle is recommended. The current pipeline default is set to use the 3-D drizzle algorithm.

• Set weighting = drizzle to use 3-D drizzle
• Set weighting = emsm to use the exponential modified Shepard's method

The FOV, slice width, number of slices, and plate scales are different for each channel. The cube_build step will use reference files to determine the proper output sampling scale as a function of wavelength. If data from a single channel or band is combined, the resulting IFU Cube will have a linearly-spaced wavelength dimension. If the user combines data covering several channels into a single data cube, the output scale corresponds to the channel with the smallest scale and the resulting IFU cube will have a non-linear wavelength dimension designed to reasonably sample the spectral line spread function (LSF) as a function of wavelength.

Cube building can be used to combine all integrations per band, channel, or combine all channels together. Association files are used to identify the desired input files for the cube building. JWebbinar 5 (MIRI and NIRSpec IFU ) gives detailed examples of how to process MIRI MRS data with the different cube building options.:

1. 12 per-band cubes: Create a composite data cube from all dithered data for each of the 12 MRS bands individually (this was the default pre-flight).
   
   
2. Channel configuration: The pipeline can also produce per-channel data cubes where all bands are combined into a single channel  (e.g., 1A, 1B, and 1C are combined into a single channel 1 cube). This is the default at the present time.
   
   

3. All-wavelength cube: The pipeline can be configured to combine all data into a single cube spanning the wavelength range of the data taken. If combining data from all 4 channels, this single cube will have a wavelength range of 4.9 to 27.9 μm. This can be useful to get an overall view of a given scene, but is extremely computationally demanding to produce, and will contain artifacts at long wavelengths resulting from sampling the output data cubes on scales far below the native detector/slicer sampling. 

   calwebb_spec3.cube_build.output_type = 'multi'

Cube building caveats

• All-wavelength cubes are created with an output scale corresponding to the sampling at the shortest wavelength. This will result in artifacts in the cube at long wavelengths where the sampling of the data is significantly smaller than the native sampling.
• Creating an all-wavelength cube is extremely computationally demanding to produce and may fail if the computer does not have enough memory.
• The JWST IFUs are significantly undersampled by the slicing optics, leading to a recommendation that users dither in order to help recover spatial information lost by the native sampling. However, this cannot be entirely fixed using a small number of dither points, and reconstructing a data cube from undersampling observations thus results in an apparent spectral "ringing" in the spectrum due to resampling noise as the spectral trace crosses detector pixel boundaries. This is inherent to the drizzle and cube building process and there is currently no correction in the pipeline for this. Further documentation of this effect is forthcoming, but the "ringing" can be mitigated by using spectral extraction apertures comparable to the PSF FWHM instead of using the spectra of individual spaxels. 
• Cube building now works for moving targets as of version 1.7.0 (2022-09-01) of the JWST pipeline. Previous versions of the pipeline early in Cycle 1 did not take into account the motion between in cube_build. 

Managing detector features

MIRI MRS has instrument specific features that can be corrected with the pipeline. These corrections are actively under development. See MIRI Features and Caveats for detailed descriptions of detector features.

Detector 1 

There is the potential for artifacts to be introduced in flight from column/row pull up/down effects produced by extremely bright sources on the detector.  Currently, detector artifacts can be best corrected by using dedicated background exposures. This effect is still under study and additional calibration is under development as results from Cycle 1 calibration are assessed.

Scattered light correction

Software documentation outside JDox: Stray Light

Scattered light is a manifestation of the MIRI cruciform diffraction at the pixel gap and reflection at the anti-reflection coating. As such the "straylight" is an integral part of the MRS PSF. Because neighboring slices on the detector do not correlate to neighboring pixels on the sky, the diffraction spikes will appear in the spectral cubes as faint sources away from the observed point source. The signal from the scattered light linked to the pixel-gap-diffraction is clearly defined in the spatial direction (detector horizontal direction) and automatically corrected in the pipeline. While this correction does a reasonable job of removing the horizontal "band" type features and reduces the amplitude of the small "spike" features, some spikes will still remain in the data. Users should be cautious about the existence of such spikes when performing PSF subtraction and/or looking for faint companions within the field of view.

Fringe correction

Software documentation outside JDox: Residual Fringe

There are 2 correction steps in the pipeline, FringeStep and ResidualFringeStep.

• FringeStep divides the detector image by a static fringe flat. The fringe flats were derived by fitting a physical model of detector fringes to observations of spatially extended sources.  Fringe flats largely remove detector fringes from MRS data, although residuals remain. For point sources and spatially semi-extended sources, the fringe pattern depends on the distance from the PSF peak (to sub-pixel accuracy). Point sources will always show residual fringes after fringe-flat correction (which are generally smallest at PSF peak).
  
  At present, the static fringe flat correction does not perform well for channel 2C resulting in fringe features remaining even after correction. This is currently being worked on and will be improved throughout Cycle 1.
  
  

• ResidualFringeStep fits sinusoids to detector-level data and removes them. Care is taken not to accidentally remove physical features. In particular, the fringe frequency is constrained to values known to match the physics of the MRS, governed by the known optical thickness of the detectors and the dichroic. The number of fringe components to be removed is governed by Bayesian evidence: the algorithm will continue to remove fringe components for as long as there's evidence for it. This residual fringe correction is not automatically run but can be applied to 2-D calibrated detector data. This step should be applied to the "rate.fits" files after photom but before the cube-building step. 

  jwst.pipeline.Spec2Pipeline.residual_fringe.skip = False

  Calibration pipeline version 1.6.2 (on 2022-07-19) fixed a bug that caused the residual fringe step to not run properly on data from the MIRIFULONG detector (channels 3 + 4).

Improved fringe flats and more sophisticated fringe-removal techniques are under development throughout Cycle 1.

References

Kavanagh, P., et al. 2023, in prep

Patapis, P., et al. 2023, in prep

Wright, G., et al. 2023, in prep