Features and caveats specific to MIRI MRS data processing in the JWST Science Calibration Pipeline are described below. This article is not intended as a how-to guide or as full documentation of individual pipeline steps, but rather to give a scientist-level overview of the calibration and any known issues that users should be aware of for their science.

An example Jupyter notebook that users can use to reprocess data with the pipeline is available on GitHub. This notebook is designed for use with any generic observing program, and illustrates how to set optional reprocessing flags that may be desired.

Summary of specific MIRI MRS pipeline issues

The information in this table about MIRI MRS pipeline issues is excerpted from Known Issues with JWST Data Products. 

Recent changes

June 2023:

• Major update to all reference flat field, static fringe flat, and photometric calibration reference files based on analysis of Cycle 1 calibration data. Minor updates to Ch 1AB wavelength solutions.
• New reference files paired with the jwst 1.11.x software allows for automatic correction of the MRS time-dependent throughput loss.
• jwst 1.11.x includes options for 1-D residual fringe correction, auto-centroiding of point source spectral extraction regions, and improved outlier detection.

MIRI MRS calibration

Astrometric calibration

The astrometric calibration of the MIRI MRS is described in detail by Patapis et al. 2023. In brief, each of the 12 bands was calibrated using a combination of ground testing and in-flight observations of the bright star 10 Lac using a 57-point raster scan that moved the star throughout the field of view.

When using target acquisition (TA), the astrometric alignment of the 12 MRS bands is generally good to about 0.03'' (1-σ radial) based on analysis of multiple point sources observed throughout Cycle 1. As illustrated in Figure 1, this is driven by the repeatability of the MRS dichroic & grating wheel assembly (DGA). This alignment is generally sufficient for most scientific analyses at 1/10 to 1/30 of the PSF FWHM, but can nonetheless be noticeable in spectra of individual data cube spaxels on the shoulder of bright point sources, for which the astrometric jump between bands can produce discontinuous jumps between bands. Likewise, this non-repeatability in the alignment means that programs that rely upon detailed modeling of the PSF should use data cubes produced for individual bands. Without target acquisition, the pointing accuracy of the MRS is 0.45" (1-σ radial), again based upon analysis of multiple point sources observed throughout Cycle 1.

The overall accuracy of the WCS embedded into MIRI MRS data products is typically 0.3" (1-σ radial), with some known cases in which the WCS was incorrect by well over an arcsecond. This is true even in cases where target acquisition was performed, such that the source can be properly centered in the field of view but the coordinates embedded into the data products are incorrect. This is driven by a combination of uncertainties in the guide star catalog, misidentified guide stars during observations, and uncertainty in the spacecraft roll angle at the large separation of the MRS from the fine guidance sensors (FGS). This in turn can produce errors in the locations used by the pipeline for extracting 1-D spectra from the data cubes. Simultaneous imaging with the MIRI imager can be used to refine the astrometry of the MRS if bright sources with Gaia positions fall within the imaging field of view, although this functionality is not yet included in the JWST pipeline.

Wavelength calibration

As discussed by Argyriou et al. 2023, the wavelength calibration of the MIRI MRS has been revised multiple times since commissioning to take advantage of ever more detailed calibration information. The original commissioning wavelength solution (distortion version FLT-2) was based upon small adjustments from the pre-flight wavelength solution and had errors of up to 0.04 μm (~ 1000 km/s) in some bands as well as an offset in some slices around 17.4 μm resulting in splitting of emission lines crossing these slices. This solution was improved early in Cycle 1 using observations of bright nebular emission lines from planetary nebula NGC 6543 to rederive an entirely flight-based solution.  The corresponding FLT-4 solution (implemented in CRDS context jwst_0970.pmap in September 2022) had a typical accuracy of 10–20 km/s throughout the MRS wavelength range but could be in error by as much as 70–80 km/s near the ends of individual bands where no nebular lines were available to constrain the solution.

FLT-5 (implemented in CRDS context jwst_1082.pmap in May 2023) radically revised the approach to MRS wavelength calibration, using observations of molecular features in the the giant planets Jupiter and Saturn paired with NEMESIS atmosphere models to constrain the wavelength solution throughout the FOV. This calibration dramatically improved the wavelength calibration throughout bands 1A–3B, as illustrated in Figure 2 (see details provided by Argyriou et al. 2023 and Harkett et al. 2023). Channels 3C–4C could not be constrained by these giant planet observations, but were updated at the same time to fold in additional information provided by planetary nebula NGC 7027 and the Be stars HD 76534 and HR 2787 (Figure 3). The typical calibration accuracy of FLT-5 is a few km/s in bands 1A–3B, and roughly 30 km/s in bands 3C–4C.

Additional minor updates provided in FLT-6 (implemented in CRDS context jwst_1094.pmap in June 2023) further adjusted the Channels 1A and 1B wavelength solutions to adjust the across-slice wavelength tilt at the few km/s level.

There are 2 ways that the wavelength solution can be given for MIRI MRS data cubes and extracted spectra. Per-band or per-channel cubes are constructed using a linear wavelength solution, the parameters of which are given using the usual FITS CRVAL, CDELT, etc., header keywords. FITS headers use a 1-based counting convention while Python uses a 0-based convention, so the wavelength solution using NumPy for a given data cube would be:

wave = (np.arange(hdr['NAXIS3']) + hdr['CRPIX3'] - 1) * hdr['CDELT3'] + hdr['CRVAL3']

Multi-channel cubes use a nonlinear wavelength solution that cannot be described by FITS header keywords alone. These cubes use the WAVE-TAB convention in which the wavelength coordinates are stored in a FITS table extension.

Note that the wavelength solutions reported by the pipeline have already been corrected for spacecraft motion and are given in the barycentric vacuum rest frame.

Photometric calibration

The initial first-year photometric calibration of the MRS (as described by Argyriou et al. 2023) was based on commissioning observations of the A7Vm star HD 163466. These commissioning observations, however, used only a single star and had limited SNR (particularly in the longer wavelength bands). Continued observations of additional stars throughout the Cycle 1 observing program permitted an updated calibration (implemented in CRDS context jwst_1094.pmap in June 2023) using multiple stars. The primary flux calibrator for the reference files delivered to CRDS in June 2023 is the O 9 V star 10 Lac, which has the highest SNR of all standard stars observed to date. Flux calibration is achieved by comparing the pipeline-extracted 1-D spectra of 10 Lac against BOSZ stellar atmosphere models (Bohlin et al. 2017). Note that there is a nearly 10% systematic uncertainty in the overall flux calibration depending on which standard star is used to determine the overall calibration (Figure 4), the origin of which is under investigation.  The August 2023 calibration update switches the primary calibrator longward of 22 µm to the G3V star 16 CygB, with a normalization offset such that it agrees with the 10 Lac calibration at shorter wavelengths.  Longward of 26 µm the August 2023 calibration is patched using observations of red targets NGC 7027 and 515 Athalea to compensate for systematic background-subtraction biases arising from the blue calibration stars becoming extremely faint at these wavelengths.

The photometric calibration is most reliable below about 20 µm, while at longer wavelengths the rising thermal background complicates efforts to compare across multiple calibration targets. While recent updates extend the MRS long wavelength cutoff to 28.7 µm, the wavelength range longward of 27.9 µm is particularly uncertain as the effective system throughput is extremely low. Further study and observations will permit further improvements of the MRS long wavelength photometric calibration throughout Cycle 2.

The MRS photometric calibration is additionally complicated by a time-dependent evolution in the effective count rate registered by the instrument that was observed during the first year of operations (see JWST Observer). As illustrated by Figure 5, this loss is most pronounced in channel 4, reaching approximately 50% over the first year of operations in band 4C. The origin of this evolution is under investigation, but the effective loss at all wavelengths can be well described by an asymptotically decaying function of time. This time-dependent evolution is taken into account in JWST pipeline versions 1.11.0 and greater, and in the JWST Exposure Time Calculator released for Cycle 3.

See also sections below on:

• MRS spectral leak
• MRS 24 µm artifact

Point vs extended sources

As with ordinary imaging observations, the surface brightness in the MRS data cubes is a convolution of the intrinsic surface brightness profile of an object with the beam of the instrument. The surface brightness measured for extended sources will thus depend on the PSF, with larger PSF widths (e.g., at longer wavelengths) resulting in lower peak surface brightnesses.

In the limit of an infinitely-extended uniform slab, PSF losses outside of a given region will be exactly offset by gains into that region. In the limit of spatially unresolved point sources, spectral extraction must take into account the fraction of light lost outside a given aperture as a function of wavelength. The MRS photometric calibration is tied to such point source observations, and will give properly calibrated results in either of these two regimes so long as the pipeline aperture correction factors are applied to extracted point source spectra.

In the intermediate regime of finite-size extended objects such correction factors will be complex and scene-dependent. Users are therefore encouraged to forward model their data if precise spectrophotometric results are required, constructing a 3-D RA/Dec/wavelength model of their scene, convolving it with the wavelength-dependent MRS PSF, and dialing the brightness of the model until the PSF-convolved model is a good match to the observations.

MIRI MRS artifacts

Cosmic ray showers and column striping

As discussed in MIRI Instrument Features and Caveats, the MIRI detectors experience cosmic ray "showers" that can produce large cosmetic artifacts in the rate images produced by the JWST pipeline. These showers are thought to result from large clusters of energetic particles produced by cosmic rays. Unlike traditional cosmic rays, which affect a small number of pixels and produce a nearly delta-function jump in the ramp for a given pixel, these cosmic rays showers can affect hundred of pixels (see Figure 6), release charge slowly over a period of many seconds, and can produce latents visible in the next integration. As a result, these shower artifacts are difficult to detect and flag in the pipeline. These showers are found in both the SHORT and LONG detectors of the MRS, and can effectively limit the depth of long observations.

Cosmic ray showers can be mitigated by dithering and taking at least 3 integrations, while 5 or more will provide better results. However, this may not be possible in general since (for pixels not affected by showers) better SNR is achieved using a small number of relatively long ramps rather than multiple short ramp exposures. Additional parameters have also been introduced into the 'jump' step of the JWST pipeline that can help to flag and remove these features (see in particular the find_showers keyword).

Also visible in Figure 6 (particularly for the SHORT detector) are stripe patterns in the vertical direction produced by drifts in the effective detector dark current and reset level with time. In brief, reference darks subtracted by the pipeline do not succesfully remove all dark-like signal from individual frames as these are time-variable. While an overall pedestal count rate for each detector image is subtracted in the straylight step by measuring the median count rate in the unilluminated region in the center of the detector between the dispersed MRS channels, this correction cannot compensate for column-dependent changes in the effective dark. The most reliable method for subtracting these column stripes is to measure their strength in a dedicated background observation taken close in time to the science observations, and subtract the result from both science and background observations.  This correction is not a part of standard pipeline processing, but may be helpful for programs attempting to reach the MRS background noise floor.

Cross artifact

Scattered light is an integral part of the MRS PSF, which is a telescope diffraction-limited PSF convolved by the MRS detector response function to incoming illumination. The scattered light in the MRS is caused by photon scattering inside of the MRS detector substrate. As discussed by Argyriou et al. 2023, the scattering occurs between the detector pixel metalization and the anti-reflection coating on the back surface of the detector (the MRS detectors are backside illuminated) and is most pronounced in channel 1.

This photon scattering in the MRS detectors manifests in the data in 2 ways: 

1. A broader PSF FWHM at short wavelengths compared to diffraction-limited PSF predictions.
2. A secondary diffraction at narrow gaps between the pixels, which act as narrow slits in the mid-infrared. This superimposes a traditional Airy diffraction pattern on the detector that is centered on the PSF. 

Because neighboring slices on the detector do not correlate to neighboring pixels on the sky (and are similarly offset in wavelength), the wings of the Airy diffraction pattern will appear in the MRS 3-D-reconstructed spectral cubes as faint sources far away from the observed point source both spatially and spectrally.  This straylight signal is automatically removed in the pipeline for all MRS observations by convolving the observed detector scene with a model of the known cruciform profile (Figure 7), and does a reasonable job of removing both the horizontal "band" type features and the small "spike" features. However, the features may be either over- or under-subtracted at the percent level, and users should thus be cautious about the existence of such spikes when performing PSF subtraction and/or looking for faint companions within the field of view.

A faint scattered light component has also been observed in the spectral direction (detector vertical direction) as well. This component is difficult to disentangle from the spectral continuum of a source, and can manifest as either a slight broadening or faint secondary peaks in the spectral line spread function (LSF). At present, this component is still under investigation and no correction is currently available. Likewise, the performance of the spatial correction degrades in the vicinity of extremely bright emission lines, and can lead to artifacts in adjacent regions on the detector.

Fringing

As discussed in depth by Argyriou et al. 2020, like most IR spectrometers the MRS experiences strong spectral fringing of order 10–30% of the incident spectral baseline at all wavelengths. These large-amplitude sensitivity modulations are caused by constructive and destructive interference produced by multiple internal reflections within different layers of the MIRI detectors. On the LONG wavelength detector, there is an additional low amplitude high frequency modulation attributed to fringing within the dichroic optical elements. While the fringe frequencies are well known, amplitudes can vary due to beating between the different fringe components and additionally are sensitive to the detailed location and intensity of objects within a given astronomical scene. Fringing thus cannot be corrected in its entirety for an arbitrary astronomical scene without forward modeling.

Multiple pipeline steps exist, however, to mitigate the impact of this fringing on science spectra and these steps generally suffice to reduce the fringe signal to below a few percent of the target flux.

The first step, applied by default in the JWST pipeline, is to divide the uncalibrated rate image by a static fringe flat constructed from observations of a bright source that fills the entire MRS field of view. This step generally does a good job of removing the strongest fringes from an astronomical scene, particularly for nearly-uniform extended sources (see Figure 8). Since the fringe signal is different for point sources however (and varies as a function of the location of a point source within the FOV) the static fringe flat cannot fully correct such objects and the default high level data products on MAST will therefore still show appreciable fringes.

The pipeline also includes 2 optional residual fringe correction steps whose purpose is to find and remove signals whose periodicity is consistent with known fringe frequencies (set by the optical thickness of the detectors and dichroics) using a Lomb-Scargle periodogram (see details given by Kavanagh et al. 2023). The number of fringe components to be removed is governed by a Bayesian evidence calculation.  The first of these residual fringe correction steps is a 2-D correction that can be applied to the flux-calibrated detector data in the calwebb_spec2 pipeline by enabling the residual_fringe step.  The second of these residual fringe correction steps is a 1-D correction (in the jwst pipeline version 1.11.0 and later) that can be applied to one-dimensional spectra extracted from MRS data cubes by setting the optional keyword extract_1d.ifu_rfcorr = True. Empirically, the 1-D correction step has been found to both work better than the 2-D correction step and is much faster for the pipeline as well.

As illustrated in Figure 9, the residual fringe correction step can work well to remove periodic fringes from the spectra of bright point sources. However, its utility is nonetheless limited in a few key ways. First, since the fringe frequencies change with wavelength, the 1-D residual fringe correction performs best when applied to spectra extracted from a single one of the 12 MRS bands at a time and does not generally work when applied to full 5–28 µm spectra across the entire MRS wavelength range. Second, the step relies upon being able to measure periodic amplitude modulations in the data. If a given spectrum is too faint, or if it contains too much astrophysical structure to reliably isolate the fringe signal, the residual fringe routine will not perform well. Finally, if the science target contains genuine astrophysical structure with frequencies similar to the known fringe frequencies (e.g., as is the case for some icy debris disks) then the residual fringe routine can remove this astrophysical signal from the data.

Work is underway during Cycle 2 to explore additional corrections for fringing (e.g., using finely sampled templates of point sources stepped across sub-pixel samples), and further updates will be provide as they become available.

Resampling noise

As discussed in MIRI MRS PSF and Dithering, the MRS is spatially undersampled by a factor of about 2 compared to the Nyquist sampling necessary to fully reconstruct the PSF delivered by the JWST optics. As a result, when the cube building software resamples the native detector pixel data into a regular cube grid there are artifacts produced by the resampling for unresolved point sources. This issue is explored in detail by Law et al. 2023.  In brief, the size of the artifact is related to the pixel phase of the dispersed trace centroid; since the trace centroid curves across the detector these artifacts appear as sinusoidal variations in the reconstructed intensity whose frequency varies with wavelength according to the rate at which the trace crosses pixel boundaries.

Figure 10 shows the theoretically expected resampling artifacts for MRS channel 1A based on a model of the JWST PSF and MIRI MRS detector sampling; these are similar to actual artifacts observed in the spectra of bright point sources with data obtained during Cycle 1. In both simulated and observed cases, the resampling noise is significantly reduced by obtaining data using a 4-point dither pattern designed to properly sample the JWST PSF. Even for a 4-point dither pattern however, few-percent artifacts remain if attempting to plot the spectra of individual spaxels from the data cubes.  Essentially, doing so is akin to a 2-D case of comparing the brightness of multiple stars throughout an undersampled image by looking only at the brightest central pixel values. As in the 2-D case, extracting photometric information from an aperture comparable to the PSF FWHM in size reduces the amplitude of the artifacts to well below 1%, ensuring flux conservation in an integrated sense.

In large part, such resampling artifacts are unavoidable as they result directly from the undersampling of the MRS optics. However, work is currently ongoing to study whether some science cases can admit corrections that enforce smoothness on sub-PSF scales, either by forward modeling the detector data or applying empirical post-facto corrections based on the known phase crossing frequency of the dispersed point source traces. Further information will be provided here as it becomes available.

Spectral leak

The MIRI MRS filters are designed to keep out-of-band light from interfering with the desired first order wavelengths dispersed in a given band. However, around 12.2 µm (channel 3A) a few-percent spectral leak admits second-order light from 6 µm (channel 1B) into the bandpass. In the commissioning-based version of the photometric calibration reference files, this spectral leak was ignored. As a result, the photometric responsivity vector effectively baked in the leak signal from the A-type star used for calibration, meaning that the pipeline would correct for the leak for objects of A-type spectral shape, produce an excess for bluer objects, and produce a dip for redder objects (see Figure 11). The photometric calibration produced during Cycle 1 (implemented in CRDS context jwst_1094.pmap in June 2023) now corrects for this spectral leak during the analysis of the spectrophotometric standard stars, meaning that the spectra produced by the pipeline contain additional flux around 12.2 µm that is only proportional to the object flux at 6 µm. As a result, very red targets with little flux around 6 µm now show no evidence of the leak, while very blue targets show a pronounced 12.2 µm artifact (Figure 11).  A future version of the pipeline will contain an optional step to correct this feature in the extracted channel 3A spectrum for a given target using the channel 1B spectrum of that target (if available). Note that since the channel 1B FOV is smaller than that for Ch3A no such correction is possible in general for extended sources that fill the entire FOV.

24 µm artifact

While the June 2023 update to the MRS photometric calibration substantially improved the reliability of the channel 4 spectra, some artifacts remained. In particular, disagreement between observations of individual calibration stars in the 24 µm regime led to an artifact that produced a spurious 10% absorption feature centered around 24.5 µm (at the long wavelength end of channel 4B). This artifact was particularly noticeable in spectra of object with otherwise smooth continua (Figure 12). This feature has been largely fixed by an update to the photometric calibration in August 2023 (CRDS context 1108).

Additional detector artifacts

A variety of additional artifacts are described on the MIRI Instrument Features and Caveats page that can affect certain programs. These include:

1) A brighter-fatter effect in which charge from the saturated cores of bright sources spills into neighboring pixels, biasing their ramps are resulting in poor fits to the resulting count rate slopes. This can affect programs that rely upon detailed modeling of the MRS PSF. See additional discussion by Argyriou et al. 2023B.

2) Column/row pull up/down effects produced extremely bright sources on the detector. This effect can result in large bands of biased signals on the detector in the rows and/or columns surrounding regions of extremely bright emission. No correction is currently available in the pipeline.

3) Warm pixels. The number of unreliable pixels has been observed to evolve over time, sufficiently fast that static bad pixel masks cannot be updated rapidly enough to account for all bad pixels present in any given science observation. This can be mitigated using dedicated backgrounds, and using those backgrounds outside the JWST pipeline to flag bad pixels in both background and science exposures. Since jwst pipeline version 1.11.0, the revised outlier detection routine has also been modified to identify such unreliable pixels as well as they are the dominant source of spurious "spikes" in extracted 1-D spectra.

MIRI MRS pipeline notes

The MIRI MRS pipeline is complex, and there are multiple points that users should keep in mind for individual stages that can affect the quality of the final data products.

WCS Accuracy

As discussed in the Known Issues with JWST Data Products, the accuracy of the world coordinate system embedded into pipeline data products is limited by the accuracy of the JWST guide star catalog and the proper identification of guide stars, particularly in crowded fields.  Since the MIRI MRS is far away from the Fine Guidance Sensors (FGS) in the JWST focal plane, uncertainties in the spacecraft roll constrained by the star trackers can also contribute to uncertainties in the global WCS. These uncertainties apply even in the case for which target acquisition was performed and the science target correctly placed at the intended location within the instrument. For the MIRI MRS, the typical WCS uncertainty is 0.3" RMS, although cases of offsets greater than 1" have been observed.

In order to mitigate this uncertainty, it is recommended where possible for users to obtain MIRI imaging simultaneously with their MRS observations. If sources with known Gaia positions fall within the MIRI imaging FOV these can be used to refine the WCS of both imaging and MRS data. A notebook demonstrating this procedure will be made available during Cycle 2.

Note that this WCS problem also affects spectral extraction.

Source type

This step determines whether the pipeline should consider the observation to be a point source or an extended source, which affects whether 1-D spectral extraction is later done from the entire FOV or using aperture photometry around a given point.

• If EXTENDED = YES is selected in the APT target description used when designing the observations, the pipeline will treat it as an extended source. 
• If EXTENDED = NO in APT, the pipeline will treat it as a point source.  
• If EXTENDED = UNKNOWN, the pipeline will try to use the dither pattern to figure out what the likely source type is. A point source-optimized dither pattern will result in "point", otherwise it will be "extended".  
• This step can be manually overridden to specify either case.
• Time-series observations will require a separate optimal reduction process. For TSO observations the pipeline processing ends before the cube building and spectral extraction steps. (See: JWST Time Series Observations Pipeline Caveats) 

Background subtraction

The MIRI MRS sees a significant component of background light at long wavelengths, both from reflected zodiacal light and from JWST thermal emission (particularly in channel 4). As such, it is important to subtract this background signal in order to measure the astrophysical signal of interest. There are presently 3 locations at which this can happen in the pipeline:

1. Background subtraction (calwebb_spec2): This performs pixel-by-pixel background subtraction in which one exposure can be directly subtracted from another, removing a background signal but introducing noise from both the science and background exposures into the final image (especially if both suffer from large cosmic ray showers). This approach is not used by the pipeline unless manually specified to do so, and analysis of Cycle 1 data shows that it is not beneficial in most cases.
   
   
2. Master background subtraction (calwebb_spec3): This subtracts the background signal from science data using a one-dimensional master background spectrum. This master background spectrum is typically created by combining all 1-D spectra from dedicated background observations processed by the calwebb_spec2 pipeline, and broadcast to the entire 2-D detector array by interpolating to the wavelengths of each detector pixel. This broadcast array is then subtracted from the individual 2-D science frames with little to no degradation of the SNR (since it uses a 1-D spectral model produced from the sigma-clipped combination of many detector pixels rather than the values of those pixels themselves). Alternatively, users can provide a custom 1-D background spectrum, which must be specific in surface brightness units of MJy/sr. A few key notes:
   
   1. This step does not run by default unless there is a dedicated background observation properly linked to the science observations. Science data cubes for programs that did not include dedicated backgrounds will not be background-subtracted unless users create and provide a custom 1-D background spectrum to the master background routine.
   2. If data are designated as linked background observations, then extract_1d will use the entire field of view as the extraction region. The master_background step recognizes the type of background exposure and uses the appropriate "x1d" data product to produce the master background. 
   3. At present, the master background spectrum can have arbitrary spectral structure. Possible future revisions to this routine are under consideration that would enforce a background model that varies smoothly as a function of wavelength, helping to reduce noisy artifacts from cosmic ray showers that can remain even after sigma-clipping throughout the field of view.
      
      
3. Annular background subtraction during spectral extraction
   1. (calwebb_spec2): If processing any target indicated as a point source, the extract_1d step will compute a background spectrum using a sigma-clipped algorithm in an annulus surrounding the location of the source. By default, this background spectrum will be subtracted from the 1-D science spectrum extracted from the circular aperture centered on the source (accounting for the effective area of that aperture). Both the background annulus and circular aperture increase in size with wavelength.
   2. (calwebb_spec3): As in calwebb_spec2, the process of spectral extraction from a 3-D data cube for point sources by default applies a background subtraction from an annular region surrounding a point source. Even in cases for which master background subtraction was already done, the annular background subtraction should be performed since the aperture correction factors applied to the data are derived assuming that the fraction of the PSF contained within the background annulus has been subtracted from the data.

Outlier detection

Despite the jump step in the calwebb_detector1 pipeline flagging jumps due to cosmic rays in the ramps of individual pixels, outliers can nonetheless persist into the calibrated 2-D data products and cause unphysical spikes in the extracted 1-D spectra. The majority of these outliers are due to warm pixels that are not flagged in the bad pixel mask.

In standard imaging modes, the approach to outlier detection is to create a version of the final scene from each input frame, median these together to get a "clean image," and blot this back to the detector frame of individual exposures to identify any outliers. Since the MRS is severely undersampled (see MIRI MRS PSF and Dithering) this resampling both forward and backward from individual exposures results in significant artifacts around unresolved point sources. These artifacts can confuse such projection-based outlier detection algorithms (a version of which was implemented in commissioning and early Cycle 1), resulting in both poor rejection of outliers and occasional rejection of some spectral segments of genuine point sources.

In June 2023 (pipeline versions 1.11.0 and greater) a new approach to outlier detection was implemented that uses a more classical approach from CCD image processing to identify sharp features on the detector that are inconsistent with the known detector PSF. This approach does substantially better at flagging real outliers while not flagging science signal in the many possible kinds of JWST targets, and runs many times faster than the original algorithm.  While an improvement on the original routine, outlier detection is still imperfect and some outliers can remain in the data cubes and extracted spectra for some science programs. As additional improvements are tested the corresponding algorithms will be updated as necessary.

Cube building

The cube_build step combines the individual 2-D frames of dispersed IFU data to create a regularized 3-D spectral cube. By default, cube building uses a 3-D variant of the classical drizzle algorithm to apportion the flux to individual cube voxels based on the relative volumetric overlap between the output voxels and the input detector pixel footprint, although an alternative method using a flux-conserving version of Shepard's method with an exponential weighting kernel (EMSM) is also available. Both algorithms are described by Law et al. 2023 and have similar performance although the 3-D drizzle approach provides data cubes with a slightly sharper PSF since the exponential weighting inherent in the EMSM approach effectively smooths the data cubes by a small kernel function. (This smoothing can, however, also serve to partially mitigate the impacts of resampling noise). The cube building algorithm can be selected using the weighting argument to cube_build:

calwebb_spec3.cube_build.weighting = 'drizzle' or 'emsm'

The FOV, slice width, number of slices, spectral resolution, and plate scales are different for each channel. The cube_build step will use the "cubepar" reference file (e.g., jwst_miri_cubepar_0012.fits) to determine the proper defaults for the spaxel scale and wavelength sampling of a given IFU data cube (see discussion by Law et al. 2023). If building cubes from a single MRS band (e.g., 1A) or channel (e.g., 2A+2B+2C) the resulting IFU cube will have a linearly-spaced step in the wavelength dimension. Cube building can also combine together data from multiple different channels (e.g., Ch1 + Ch2), in which case the IFU cube will have a non-linear wavelength solution designed to reasonably sample the spectral LSF as a function of wavelength throughout the entire cube. In such multi-channel cases the spatial scale is set to the scale appropriate for the shortest wavelengths in the cube.  Individual band, full-channel (default for the JWST pipeline), or all-wavelength cubes can be specified using the output_type argument to cube_build:

calwebb_spec3.cube_build.output_type = 'band', 'channel', or 'multi'

While such all-wavelength "multi" cubes can be useful to get an overview of a given scene they may be of limited scientific utility since (1) Residual fringe correction cannot be applied to such spectra, (2) The spaxel scale is optimized for short wavelengths and thus unphysically oversampled at long wavelengths (dramatically increasing covariance and visual artifacts), (3) MRS grating wheel repeatability will result in small sub-pixel jumps in source centroids between bands, and (4) spectral overlap regions between bands will combine together data with different spectral resolutions.

Note that cubes are constructed in surface brightness units of MJy/sr, and will thus need to be multiplied by the spaxel (contained in the header keyword PIXAR_SR and different for each channel) in order to sum the flux from individual spaxels. By default, all voxels for which there is no valid scientific data (e.g., outside the IFU cube footprint) are given values of "NaN" so that they can be easily identified and excluded from analysis, and so that they do not artificially bias automated scaling algorithms that display the IFU cube data. Note that major FITS file viewers (e.g., ds9) have preferences that allow "NaN" values to be rendered as a unique color to distinguish them from science voxels.

The pipeline also produces both "ERR" and "DQ" data cubes corresponding to the estimated uncertainty and data quality flag of a given cube voxel. Note, however, that at present the values given in the "ERR" cubes are incorrect by a factor of 10 or more due to a bug in estimating the uncertainties upstream in the JWST pipeline; investigation is ongoing into the origins of this bug.  Note also that there is substantial covariance between adjacent voxels because of the resampling performed by cube building. Law et al. 2023 discusses this issue in greater detail and provides a series of recommendations for scaling the "ERR" values to account for covariance as well in extracted spectra. This recommendations will be incorporated into JDox once the "ERR" values are fixed.

Spectral extraction

The spectral extraction step is designed to extract 1-D spectra from the 3-D IFU data cubes. Such 1-D spectra cannot encompass the full range of structures that may be present in the data cubes, and are thus tailored to 2 specific use cases.

For extended sources, spectral extraction averages the flux within the entire MRS field of view to provide a composite spectrum in surface brightness units of MJy/sr. Since the field of view is different for each MRS spectral band (and increases substantially for longer wavelength channels) the extraction region will thus be different as well. Depending on the source geometry within the FOV, the extracted 1-D spectrum may thus be discontinuous between channels as it corresponds to a different physical area on the sky.

For point sources, spectral extraction sums the flux within a conical aperture at the target location to provide a composite spectrum in flux units of Jy. This conical aperture grows with wavelength to enclose roughly the same total fraction of the PSF, which increases in size by a factor of 5 over the MRS spectral range. The pipeline performs background subtraction for point sources using an annular region surrounding the central aperture which likewise grows with wavelength.  Since the MRS PSF extends throughout the entire field of view (especially at longer wavelengths), the pipeline also applies an aperture correction to account for both the light lost outside the central circular aperture and the light subtracted by the annular background subtraction.  By default, the extraction region has a radius of twice the PSF FWHM at all wavelengths in order to eliminate the impact of resampling noise on the extracted spectrum.

As discussed in the WCS section above, uncertainties in the WCS embedded in the data cubes can result in spectral extraction being performed at the wrong location if using the target coordinates to specify the source location.  Figure 13, for instance, illustrates the impact that this can have on the resulting spectrum, in some cases producing negative extracted fluxes. Since pipeline version 1.11.0, it is recommended to specify

calwebb_spec3.extract_1d.ifu_autocen = True

in order to enable automated source centroiding (using the DAO Star Finding algorithm applied to an image of the cube collapsed across all wavelengths).

References

Argyriou, I., et al. 2020, A&A, 641, A150 (MRS Fringing)
The nature of point source fringes in mid-infrared spectra acquired with the James Webb Space Telescope

Argyriou, I., et al. 2023, A&A, 675, A111 (MRS Overview)
JWST MIRI flight performance: The Medium-Resolution Spectrometer

Argyriou, I., et al. 2023B, A&A submitted (MIRI brighter-fatter effect)
The Brighter-Fatter Effect in the JWST MIRI Si:As IBC detectors I. Observations, impact on science, and modelling

Bohlin, R., et al. 2017, AJ, 153, 234 (BOSZ atmosphere models)
A New Stellar Atmosphere Grid and Comparisons with HST/STIS CALSPEC Flux Distributions

Harkett, J., et al. 2023, in prep

Kavanagh, P., et al. 2023, in prep

Law, D. R., et al. 2023, AJ, 166, 45 (MRS Cube Building)
A 3D Drizzle Algorithm for JWST and Practical Application to the MIRI Medium Resolution Spectrometer

Patapis, P., et al. 2023, A&A submitted (MRS Distortion)
Geometric distortion and astrometric calibration of the JWST MIRI Medium Resolution Spectrometer

Wright, G. S., et al. 2023, PASP, 135, 048003 (MIRI Overview)
The Mid-infrared Instrument for JWST and Its In-flight Performance