JWST Post-Pipeline Data Analysis

Some of the tools available to work on fully processed data products, including a variety of community-based astronomical analysis packages and custom software developed by STScI to work specifically with the JWST data formats, are described in this article.

On this page

Available software tools

There are a variety of JWST post-pipeline data analysis tools that can assist observers in viewing and analyzing their JWST data (see Table 1). These tools have been developed by a variety of members of the astronomical community, but are generally written in Python and work with Astropy. Each package has its own installation instructions, but most can be installed with standard pip commands.

Note that Astropy is a community library, and the JWST analysis tools' success relies on community members, like you, to engage in the development process via bug reports (most effectively filed as GitHub issues, but the JWST help desk is fine as well), or by code contributions through GitHub pull requests. Use of the development versions of the code straight from GitHub comes with the following caveats: at any given time, the code may not actually run or return correct results, and the documentation may be inconsistent with the code. Users who are not interested in contributing to the development software should use fully released versions (a standard pip install). Details regarding the difference in the installation instructions can be found in the respective documentation for each package.

These post-pipeline data analysis tools can help astronomers with the often iterative and interactive workflow involved in converting science calibration pipeline data products into meaningful scientific results. This typically involves tasks such as:

  • Inspecting data and data quality information
  • Masking or flagging data and using those annotations to guide later steps in the analysis
  • Using the results of interactive analysis to guide a custom run of the pipeline (e.g., tweaking spectral extraction parameters or background estimates)
  • Performing optimized spectral extraction techniques
  • Combining data sets in various ways, with careful attention to astrometry, PSF matching, and other issues
  • Source detection and photometry using different choices or algorithms than those used in the pipeline
  • Measuring lines and continuum in spectral data
  • Fitting models to data or otherwise testing hypotheses

A typical workflow may thus involve highly interactive exploratory analysis on small portions of the data, followed by the development of custom scripts to automate the analysis on larger datasets.


Table 1. Tools for working with JWST data

Package

Purpose

GitHub RepositoryDocumentation

Astropy

A community python library for astronomy

GitHub

ReadTheDocs

glue

A Python library to explore relationships within and among related datasets

GitHub

ReadTheDocs

ginga

A toolkit designed for building viewers for scientific image data in Python

GitHub

ReadTheDocs

photutils

Tools for detecting and performing photometry of astronomical sources

GitHub

ReadTheDocs

specutilsTools for performing spectroscopic analysis of astronomical sourcesGitHubReadTheDocs
specreduceA toolkit designed to reduce and calibrate spectroscopic data.GitHub

ReadTheDocs

astroimtools

Convenience tools for working with astronomical images

GitHub

ReadTheDocs

imexam

Tool for simple image examination, and plotting, with similar functionality to IRAF's imexamine

GitHub

ReadTheDocs

jdaviz

An interactive analysis tool designed to be integrated into a Jupyter notebook science workflow.

Included ImvizSpecviz, Specviz2d, Cubeviz, and Mosviz:

  • Imviz is an image viewer with some similarities to ds9
  • Specviz is a 1D spectra analysis tool with similar functionality to IRAF's splot
  • Specviz2d is a 2D/1D spectra analysis tool designed to perform spectral extraction
  • Cubeviz is an analysis tool for 3D spectroscopy
  • Mosviz is an analysis and visualization tool for multi-object spectroscopy
GitHubReadTheDocs

asdf

Advanced Scientific Data Format is a next generation interchange format for scientific data 

GitHub

ReadTheDocs

gwcs

Generalized World Coordinate System tools for dealing with image and spectral geometries

GitHub

ReadTheDocs

synphot

Synthetic photometry toolkit for building model spectra and estimating count-rates

GitHub

ReadTheDocs



Jdaviz

The JWST Data Analysis Visualization (Jdaviz) tool is a specific component of the broader analysis tool ecosystem that has been developed by STScI to work explicitly with JWST data products using Astropy functionality. The JDAviz tool may be operated as a standalone desktop applications, within a Jupyter notebook environment, or as embedded windows within a website.

Jdaviz is built using using the glue visualization framework. For users who are interested in using glue in contexts beyond those provided for in jdaviz, please visit https://glueviz.org for download links and more information.

Jdaviz  currently has 5 available configurations:

  • Imviz is a 2D image viewer.
  • Specviz is a tool for visualization and quick-look analysis of 1D astronomical spectra. 
  • Specviz2d is a tool for visualization and extraction of 2D spectra.
  • Cubeviz provides access to and manipulation of spectroscopic data cubes (like those produced by JWST MIRI MRS or NIRSpec IFU), along with 1D spectra extractions from the cube.
  • Mosviz is a visualization tool for viewing multi-object spectrograph data (e.g., JWST NIRSpec MOS mode), and includes viewers for 2D and 1D spectra as well as contextual information like on-sky views of the spectrograph slit.

For further information see the Jdaviz ReadTheDocs page and an overview of available Jdaviz Tutorials.



JWebbinars

Starting in 2021, STScI organized and hosted a variety of community training webinars on various aspects of the JWST Science Calibration Pipeline and on post-pipeline data analysis tools. These sessions, called JWebbinars, provided users with general background and supporting materials such as Jupyter notebooks and overview PowerPoint presentations.

Information can be found on the JWebbinars page.



JWST data analysis (JDAT) notebooks

A series of example Jupyter notebooks have been built to demonstrate expected science workflows using various JWST observing modes. While focused on JWST, they show useful workflows for analyzing data from other observatories as well. The notebooks utilize Astropy packages and the Jdaviz visualization tools when possible. These notebooks can be downloaded and executed by cloning the GitHub repository to your local computer. The documentation contains detailed instructions for installing the notebooks, as well as rendered versions of the notebooks that allow you to view them in your browser without downloading. See the table on the landing page of the JDAT Notebooks documentation for a complete and up-to-date list of notebooks.




Notable updates

 

  • Updates for ERO and commissioning data public release.

 

  • Updated notebooks for Build E release.

 

  • Added section on Jdaviz, updated notebooks for Build D release, added section on JWebbinars

 

  • Added links to JDAT Jdaviz tools.

 

  • Updated Software Package and Notebook status to be consistent with Build C for Data Analysis Notebook delivery.  
Originally published