Pandeia report dictionary structure

The pandeia report dictionary contains the following keys and content:

• sub_reports: list, containing information on each exposure for multiple exposures (e.g., IFU dither patterns or coronagraphic imaging). Each report in the sub_reports list will be in the same format as a standard dictionary report.
  
  
• information: dictionary, containing information on the exposure specifications that Pandeia used to generate its result. It contains:
  
  • calc_type: string, the type of 2D projection performed. One of slitless, multiorder, image, spec
  • exposure_specification: dictionary. Contains information on how the exposure was conducted and the assumed telescope and instrument status. Contains the following:
    • exposure_time: floating point, exposure time for each exposure (sec)
    • total_exposure_time: floating point, total time spent observing the target; equal to exposure_time multiplied by the number of exposures (sec)
    • measurement_time: floating point, time between the first and last measurements of a pixel in an integration multiplied by the number of integrations per exposure (sec)
    • saturation_time: floating point, time from the reset of a pixel to the final read of that pixel in an integration multiplied by the number of integrations per exposure (sec)
    • duty_cycle: floating point, the fraction of the exposure time that was devoted to observing the target; duty_cycle = saturation_time*nint/exposure_time
    • nexp: integer, number of exposures
    • nframe: integer, number of frames per group
    • tframe: floating point, time per frame (sec)
    • nskip: integer, number of skipped frames per group; only supported by some readout patterns
    • frame0: boolean, whether the first frame was downlinked and used in the ramp fit; applies only to NIRCam
    • ngroup: integer, number of groups per integration
    • tgroup: floating point, telescope time per group (sec)
    • nprerej: integer, either 0 or 1 (default 0); applies only to MIRI; number of groups at the beginning of the ramp that were rejected by the pipeline
    • nint: integer, number of integrations per exposure
    • npostrej: integer, either 0 or 1 (default 0); applies only to MIRI; number of groups at the end of the ramp that were rejected by the pipeline
    • total_integrations: integer, number of integrations in all exposures combined
    • nramps: integer, total number of ramps in the observation; nramps is equal to nint multiplied by the number of exposures
    • readout_pattern: string, name of the readout pattern
    • subarray: string, name of subarray used for the exposure
    • det_type: string, type of detector involved (e.g., 'h2rg' for NIRCam short wavelength detectors)
    • nsample: integer, number of samples averaged when reading a single pixel; applies only to MIRI
    • tsample: floating point, time averaged when reading a single pixel; applies only to MIRI (sec)
    • nsample_skip: integer, number of samples skipped while reading a single pixel; applies only to MIRI
    • nsample_total: integer, total number of samples
    • tfffr: integer, extra time for full frame resets (sec)
      
      
• warnings: dictionary, contains all warnings generated by the exposure. Warnings can include:
  
  • <instrument>_missing_instrument_aperture: generated if the supplied configuration dictionary did not include the aperture information; the warning text will indicate what the aperture has been set to
  • partial_saturated: indicates partial saturation of at least one pixel; the warning text will include the number of partially saturated pixels
  • full_saturated: indicates full (unrecoverable) saturation of at least one pixel; the warning text will include the number of fully saturated pixels
    
    
• transform: dictionary, containing the coordinate transform information that describes the image axes for the 2-D and 3-D data. Contains the following values:

  • wave_refpix: integer, the reference pixel of the wavelength arrays (pixels)

  • wave_refval: floating point, the value of wave_refpix (μm)

  • wave_step: floating point, interval between successive points in the wavelength array (μm)
    

  • wave_size: integer, the size of the wavelength axis (pixels)

  • wave_min: floating point, the minimum wavelength value of the 2-D and 3-D data (μm)

  • wave_max: floating point, maximum wavelength of the 2-D and 3-D data (μm)

  • x_refpix: integer, the reference pixel for the x-axis in the 2-D and 3-D data (pixels)

  • x_refval: floating point, the value of the x_refpix pixel of the x-axis of the 2-D and 3-D data (arcsec)

  • x_step: the per-pixel size of the x-axis of the 2-D and 3-D data (arcsec)

  • x_size: integer, the size of the x-axis of the 2-D and 3-D data supplied in the report (pixels)

  • x_min: floating point, the minimum value of the x-axis of the 2-D and 3-D data (arcsec)

  • x_max: floating point, the maximum value of the x-axis of the 2-D and 3-D data (arcsec)

  • y_refpix: integer, the reference pixel for the y axis in the 2-D and 3-D data (pixels)

  • y_refval: floating point, the value of the y_refpix pixel of the y axis of the 2-D and 3-D data (arcsec)

  • y_step: floating point, the per-pixel size of the y-axis of the 2-D and 3-D data (arcsec)

  • y_size: integer, the size of the y-axis in the 2-D and 3-D data (pixels)

  • y_min: floating point, the minimum value of the y-axis in the 2-D and 3-D data (arcsec)

  • y_max: floating point, the maximum value of the x-axis in the 2-D and 3-D data (arcsec)

  • wave_det_refpix: integer, the detector reference pixel in wavelength space (if applicable) (pixels)

  • wave_det_refval: floating point, the value of wave_refpix on the detector (if applicable) (μm)

  • wave_det_step: floating point, the distance in wavelength space between adjacent pixels on the detector (if applicable) (μm)

  • wave_det_size: integer, the length of the wavelength axis of the detector (pixels)

  • wave_det_min: floating point, the minimum wavelength value of the detector (μm)

  • wave_det_max: floating point, the maximum wavelength value of the detector (μm)
    
    

• scalar: dictionary, contains scalar values of interest from the observation. Includes the following values:

  • exposure_time: floating point, the same value as exposure_time in the 'information:exposure_specification' part of the report (sec)

  • aperture_size: floating point, size of the extraction aperture (arcsec)

  • cr_ramp_rate: floating point, expected cosmic ray events per pixel per ramp (events/sec/pixel)

  • reference_wavelength: floating point, wavelength used for determining scalar outputs (μm)

  • extraction_area: floating points, area (in square arcseconds) from which data is extracted for scalar outputs (arcsec²)

  • all_dithers_time: floating point, total time (in seconds) spent during the observation during all exposures/dithers, whether on-target or off-target (sec)

  • total_integrations: integer, the same value as total_integrations in the 'information:exposure_specification' part of the report

  • saturation_time: floating point, the same value as saturation_time in the 'information:exposure_specification' part of the report (sec)
    

  • measurement_time: floating point, the same value as measurement_time in the 'information:exposure_specification' part of the report (sec)
    

  • fraction_saturation: floating point, the fraction describing how close to saturation the brightest pixel on the detector is
  • detector_ngroups: integer, the maximum number of groups that can be requested before the brightest pixel on the detector saturates
    

  • sn: floating point, signal-to-noise ratio for the observation

  • extracted_noise: floating point, total noise counts extracted in the background region (e^–/sec)

  • extracted_flux: floating point, total flux extracted from the extraction aperture (e^–/sec)

  • brightest_pixel: floating point, rate of brightest pixel on the 2-D image (e^–/sec)

  • duty_cycle: floating point, the same value as duty_cycle in the 'information:exposure_specification' part of the report

  • background: floating point, flux of the background spectrum at the reference wavelength (MJy/sr)

  • contamination: floating point, flux from contamination by other (non-target) sources in the background region (e^–/sec)

  • background_sky: floating point, flux in the background extraction region contributed by the sky (e^–/sec)

  • disperser: string, the grating/prism/grism in use, if any

  • filter: string, the filter in use, if any

  • y_offset: floating point, the y-offset of the target (arcsec)

  • x_offset: floating point, the x-offset of the target (arcsec)

  • background_area: floating point, the background extraction area (arcsec²)

  • background_total: floating point, the background flux including both sky and contamination (e^–/sec)

  • total_exposure_time: floating point, the same value as total_exposure_time in the 'information:exposure_specification' part of the report (sec)
    
    

• 1d: dictionary, Contains 1-D spectra representing various aspects of the observation, presented either as a function of wavelength or alongside the reference wavelength. In the descriptions below, names in italics are single values presented against the reference wavelength if the calculation is an 'imaging' calculation. Values include:

  • fp: count rate at the focal plane (μm, e^–/sec)

  • extracted_contamination: flux from contamination (μm, e^–/sec)

  • total_flux: combined target and background flux (μm, e^–/sec)

  • bg: background flux (μm, MJy/sr)

  • extracted_noise: standard deviation of the extracted flux (μm, e^–/sec)

  • bg_rate: background flux at the focal plane (μm, e^–/sec)

  • extracted_bg_only: extracted background flux not including contamination (if any) (μm, e^–/sec)

  • n_partial_saturated: number of partially saturated pixels (μm, unitless)

  • n_full_saturated: number of fully saturated pixels (μm, unitless)

  • sn: signal-to-noise ratio (μm, unitless)

  • wave_pix: single array containing the reference wavelength as its only value (μm)

  • extracted_flux: total extracted flux from the target (μm, e^–/sec)

  • extracted_flux_plus_bg: total countrate including both target and background (μm, e^–/sec)

  • wave_calc: single array containing the wavelengths over which the exposure was calculated (μm)

  • extracted_bg_total: total extracted background flux (μm, e^–/sec)

  • target: target flux as a function of wavelength (μm, e^–/sec)
    
    

• 2d: dictionary, contains 2-D images of the on-detector observation. Values include:
  • snr: 2-D NumPy array of floating point values, the signal-to-noise ratio for each pixel in the observation
  • detector: 2-D NumPy array of floating point values, the on-detector countrate for each pixel in the observation (e^–/s)
  • saturation: 2-D NumPy array of floating point values, the detector saturation information; pixels have the value of 0 (unsaturated), 1 (partial saturation), or 2 (full saturation)
  • ngroups_map: 2-D NumPy array of integer values, the maximum number of groups before a given pixel on the detector would saturate
    
    
• 3d: dictionary, contains 3-D data cubes of the on-detector observations. Available only for MIRI MRS, MIRI MRS Time Series, and NIRSpec IFU. Values include:
  • flux: 3-D NumPy array of floating point values, contains flux as a function of wavelength for each pixel at each wavelength in Pandeia's internal observation measurement (μm, e^–/s)
  • flux_plus_background: 3-D NumPy array of floating point values, contains flux and background together for each pixel at each wavelength (μm, e^–/s)
    
    
• input: dictionary, containing information about the calculation inputs. When running Pandeia from Python, this should simply be a copy of the configuration dictionary (see Pandeia Configuration Dictionaries for more information on the configuration dictionary). As such, its contents will not be further discussed here.

Pandeia report fits object

Pandeia can also output a dictionary of FITS objects, by outputting a report object (created by running "perform_calculation(<configuration>, dict_report=False)" and running the 'as_fits()' method on the resulting Report object)

• 1d: dictionary, with a FITS HDU list for every item in the regular 1-D report dictionary mentioned above. All of the FITS headers except wave_calc and wave_pix will have 2 fields in the first header: wavelength (either wave_calc or wave_pix) and the property in question.
  
  
• 2d: dictionary, with a FITS HDU list for every item in the regular 2-D report dictionary. All of the FITS headers have WCS information extracted from the detector pixel grid. These headers are, for 2-D spectroscopic modes, assumed to be linear wavelength scales; it may be necessary to use the 1-D wave_calc value to generate accurate wavelengths.
  
  
• 3d: dictionary, with a FITS HDU list for every item in the regular 3-D report dictionary. All of the FITS headers have WCS information extracted from the detector pixel grid. These headers are assumed to be linear in wavelength scale; it may be necessary to use the 1-D wave_calc values to generate accurate wavelengths.

Pandeia report object

The pandeia 'Report' object contains the same data as the report dictionary (and can be converted into the report dictionary via its 'as_dict()' method, or fits dictionary via its 'as_fits()' method), but also contains some additional information.

Per-pixel background count rate

The Pandeia Report object stores the per-pixel background count rate in the internal property 'bg_pix'. This contains a 2-D array of the entire simulated region of the detector with the per-pixel sky background for each pixel, and could potentially be useful in determining background flux in order to compare observations when looking for the lowest possible background.

Interpreting downloads from the JWST ETC website

When running simulations on the JWST ETC website, you can download the result of a particular simulation. In addition to the input values used for the simulation (discussed in the Pandeia Quickstart, the download also contains many FITS files which provide information about the simulation, and which often correspond to the content of a Pandeia report.

Structure of a JWST ETC result download

JWST ETC downloads contain the following files and folders. Note that these files are also discussed on the JWST ETC Downloads page.

• "backgrounds.fits": FITS file containing the spectrum used to determine background count rates
• cube (non-IFU observations)
  • cube_flux_plus_background.fits
  • cube_flux.fits
  • model (empty folder)
• cube (IFU observations)
  • cube_reconstructed.fits
  • cube_reconstructed_noise.fits
  • cube_reconstructed_snr.fits
  • cube_reconstructed_saturation.fits
  • cube_reconstructed_signal.fits
  • model
    • cube_flux_n.fits (one for each IFU element, with the n replaced by the element number)
    • cube_flux_plus_background_n.fits (one for each IFU element, with the n replaced by the element number)
• image
  • image_detector.fits
  • image_ngroups_map.fits
  • image_saturation.fits
  • image_snr.fits
• input.json (discussed in the Pandeia Quickstart)
• lineplot
  • lineplot_bg_rate.fits
  • lineplot_bg.fits
  • lineplot_extracted_bg_only.fits
  • lineplot_extracted_bg_total.fits
  • lineplot_extracted_contamination.fits
  • lineplot_extracted_flux_plus_bg.fits
  • lineplot_extracted_flux.fits
  • lineplot_extracted_noise.fits
  • lineplot_fp.fits
  • lineplot_n_full_saturated.fits
  • lineplot_n_partial_saturated.fits
  • lineplot_sn.fits
  • lineplot_target.fits
  • lineplot_total_flux.fits
  • lineplot_wave_calc.fits
  • lineplot_wave_pix.fits

Accessing downloaded FITS files

FITS files are most easily accessed via the AstroPy module, in particular "astropy.io.fits". Below is an example of obtaining data from a lineplot file, assuming that Python is run from the base directory of the download.

import astropy.io.fits as f
 
with f.open('lineplot/lineplot_extracted_bg_total.fits') as input_file:
    reference_wavelength = input_file[1].data[0]['WAVELENGTH']
    extracted_flux = input_file[1].data[0]['extracted_bg_total']

Correspondences with the Pandeia report dictionary

Downloaded background file

The "backgrounds.fits" file included in the download contains a data table in its first extension. This table includes the following columns:

• wavelength: the wavelength in μm
• background: total background flux
• thermal: the thermal component of the total background flux
• straylight: the stray light component of the background flux
• infield: the sky component of the background flux

Sample code

Displaying an image of the signal-to-noise ratio of a Pandeia imaging observation

# The following section is only needed if the PYSYN_CDBS environment variable is not already set.
# The PYSYN_CDBS environment variable should point to the path of the CDBS data files
import os
location_of_cdbs = "/path/to/cdbs/files"
os.environ['PYSYN_CDBS'] = location_of_cdbs
# End section

# The following section is only needed if the pandeia_refdata environment variable is not already set
# The pandeia_refdata environment variable should point to the path of the pandeia reference data
import os
location_of_pandeia_refdata = "/path/to/pandeia/refdata"
os.environ['pandeia_refdata'] = location_of_pandeia_refdata
# End section
import matplotlib.pyplot as plt
from pandeia.engine.calc_utils import build_default_calc
from pandeia.engine.perform_calculation import perform_calculation

# The following are parameters which can easily be changed
telescope = 'jwst'
instrument = 'nircam'
mode = 'sw_imaging'

# Source parameters
offsets = {'x': 0., 'y': 0.}
orientation = 27.
geometry = 'gaussian2d'
major_axis = 8.5 # arcseconds
minor_axis = 0.75 # arcseconds
name = 'Blackbody'
sed = 'blackbody'
temp = 50000.
bandpass = 'bessell,j'
magnitude = 18.

configuration = build_default_calc(telescope, instrument, mode)
scene = {}
scene['position'] = {'position_parameters': ['x_offset', 'y_offset', 'orientation']}
scene['position']['x_offset'] = offsets['x']
scene['position']['y_offset'] = offsets['y']
scene['position']['orientation'] = orientation
scene['shape'] = {'geometry': geometry, 'major': major_axis, 'minor': minor_axis, 'norm_method': 'integ_infinity', 'surf_area_units': 'arcsec^2'}
scene['spectrum'] = {'name': name, 'spectrum_parameters': ['sed', 'normalization']}
scene['spectrum']['sed'] = {'sed_type': sed, 'temp': temp}
scene['spectrum']['normalization'] = {'type': 'photsys', 'norm_fluxunit': 'abmag'}
scene['spectrum']['normalization']['bandpass'] = bandpass
scene['spectrum']['normalization']['norm_flux'] = magnitude
scene['spectrum']['lines'] = []
scene['spectrum']['extinction'] = {'bandpass': 'j', 'law': 'mw_rv_31', 'unit': 'mag', 'value': 0}
configuration['scene'][0] = scene

report = perform_calculation(configuration)

plt.imshow(report['2d']['snr'])
plt.colorbar()
plt.show()

Note that the above is being done in a very simplistic way, and that Matplotlib, in particular, has many options that aren't on display here. See the matplotlib user's guide for many more details.

Displaying the signal-to-noise ratio of a spectrum as a function of wavelength

# The following section is only needed if the PYSYN_CDBS environment variable is not already set.
# The PYSYN_CDBS environment variable should point to the path of the CDBS data files
import os
location_of_cdbs = "/path/to/cdbs/files"
os.environ['PYSYN_CDBS'] = location_of_cdbs
# End section

# The following section is only needed if the pandeia_refdata environment variable is not already set
# The pandeia_refdata environment variable should point to the path of the pandeia reference data
import os
location_of_pandeia_refdata = "/path/to/pandeia/refdata"
os.environ['pandeia_refdata'] = location_of_pandeia_refdata
# End section
import matplotlib.pyplot as plt
from pandeia.engine.calc_utils import build_default_calc
from pandeia.engine.perform_calculation import perform_calculation

# The following are parameters which can easily be changed
telescope = 'jwst'
instrument = 'nirspec'
mode = 'fixed_slit'

# Source parameters
offsets = {'x': 0., 'y': 0.}
geometry = 'gaussian2d'
major_axis = 8.5 # arcseconds
minor_axis = 0.75 # arcseconds
name = 'G2V Star'
sed = 'phoenix'
key = 'g2v'
bandpass = 'sdss,r'
magnitude = 18.

configuration = build_default_calc(telescope, instrument, mode)
scene = {}
scene['position'] = {'position_parameters': ['x_offset', 'y_offset']}
scene['position']['x_offset'] = offsets['x']
scene['position']['y_offset'] = offsets['y']
scene['shape'] = {'geometry': 'point'}
scene['spectrum'] = {'name': name, 'spectrum_parameters': ['sed', 'normalization']}
scene['spectrum']['sed'] = {'sed_type': sed, 'key': key}
scene['spectrum']['normalization'] = {'type': 'photsys', 'norm_fluxunit': 'abmag'}
scene['spectrum']['normalization']['bandpass'] = bandpass
scene['spectrum']['normalization']['norm_flux'] = magnitude
scene['spectrum']['lines'] = []
scene['spectrum']['extinction'] = {'bandpass': 'j', 'law': 'mw_rv_31', 'unit': 'mag', 'value': 0}
configuration['scene'][0] = scene

report = perform_calculation(configuration)

disperser = configuration['configuration']['instrument']['disperser']
fig, ax = plt.subplots()
ax.plot(report['1d']['sn'][0], report['1d']['sn'][1])
ax.set(xlabel='Wavelength (micron)', ylabel='SNR', title='G2V Star observed with NIRISS {}'.format(disperser))
plt.show()