JWST Post-Pipeline Data Analysis

JWST post-pipeline data analysis tools will be used for viewing and analyzing JWST data. While its development is still ongoing, overviews, software installation, and training materials are currently available for interested users.

On this page

JWST post-pipeline data analysis tools are distributed as part of AstroConda to assist observers in viewing and analyzing their JWST data. The tools are generally written in Python and work with Astropy. Development is ongoing1

The suite of post-pipeline data analysis tools is intended to help astronomers with the often iterative and interactive workflow involved in converting these pipeline data products into meaningful scientific results. This involves tasks such as:

  • inspecting data and data quality information;
  • masking or flagging data and using those annotations to guide later steps in the analysis;
  • using the results of interactive analysis to guide a custom run of the pipeline (e.g., tweaking spectral extraction parameters or background estimates);
  • combining data sets in various ways, with careful attention to astrometry, PSF matching, and other issues;
  • source detection and photometry using different choices or algorithms than those used in the pipeline;
  • measuring lines and continuum in spectral data;
  • fitting models to data or otherwise testing hypotheses.

A typical workflow involves highly interactive exploratory analysis on small portions of the data, followed by the development of custom scripts to automate the analysis on larger data sets.

1  All software is open source and community contributions are welcome in the form of suggestions, bug reports, or actual code. Further details on how to contribute can be found at the Data Analysis Tools Development Forum.



Software installation

The recommended way to install stable versions of the JWST data analysis tools is to use AstroConda

There are development versions of many of the tools linked in the "Repository" column in Table 1 that link to the open-source development locations on Github. Contributions to these developments are welcomed via bug reports (most effectively filed as github issues, but the JWST help desk is fine as well), or by code contributions through github pull requests. Use of the development versions of the code straight from github comes with the following caveats: at any given time, the code may not actually run or return correct results, and the documentation may be inconsistent with the code. Users who are not interested in contributing to the development software should use the versions in AstroConda.



Software packages

Table 1 provides links to further information about the tools. User training workshops are being offered to help new users familiarize themselves with these tools.


Table 1.  Software tools information

Package

Purpose

Repository

Maturity

Astropy

A community python library for astronomy (documentation & tutorials)

Astropy

 

glue

A Python library to explore relationships within and among related datasets.

glue

 

ginga

A toolkit designed for building viewers for scientific image data in Python.

ginga

 

Photutils

Tools for detecting and performing photometry of astronomical sources.

Photutils

astroimtools

Convenience tools for working with astronomical images.

astroimtools

 

imexam

Tool for simple image examination, and plotting, with similar functionality to IRAF's imexamine.

imexam

specviz

An interactive astronomical 1D spectra analysis tool with similar functionality to IRAF's splot.

specviz

mosviz

A quick-look analysis and visualization tool for multi-object spectroscopy.

mosviz

 

cubeviz

Interactive analysis tool for 3D spectroscopy (coming soon)

cubeviz

 

asdf

Advanced Scientific Data Format is a next generation interchange format for scientific data (docs)

asdf

gwcs

Generalized World Coordinate System tools for dealing with image and spectral geometries (docs)

gwcs

synphot

Synthetic photometry toolkit for building model spectra and estimating count-rates. (docs)

synphot


Levels of maturity run from prototypes with little or no documentation, symbolized by buds, , progressing through various levels:    to 

The exact meanings of these icons are a bit hard to quantify, but the flowers will tend to be still lacking in documentation and important features. The cherries are generally quite robust and well documented. The cherry pies are ready to be baked into your day-to-day workflow. 

Be aware that all of the packages above are in very active stages of development, including Astropy and glue. For the ones at the cherry pie level, there is significant attention given to backwards compatibility as the APIs of the different modules evolve.



Training resources

There are many resources available for learning Python and for using Python for astronomical data analysis. This section provides pointers to astronomy-focused materials and to more JWST-specific training materials.

General python astronomical data analysis training


Table 2. List of training resources for python astronomical data analysis

Resource

Written materials

Videos

Comments

Using Python for Astronomical Data Analysis

Notebooks


January 2017 American Astronomical Society Special Session

Astropy

Tutorials



Scientific Python Course at STScI

Notebooks

Videos

Notebooks are from 2015 version of course; videos from 2012-13

Practical Python for Astronomers

Web documents


2011, 2012, 2013 Smithsonian Astrophysical Observatory

JWST focused training materials

Training materials for JWST data analysis will eventually include worked examples of common workflows, using outputs from the JWST pipeline. Currently the materials are more general than that, and make use example data sets from other observatories. Table 3 provides links to these materials.


Table 3. List of JWST focused training materials

ResourceWritten materialsVideosComments
JWST User Training in Data AnalysisNotebooksWebcast2016 Workshop at STScI


Table 4 shows a more topic-oriented map of the materials from the 2016 Workshop.


 Table 4. Introductory material

Introductory lectures
PythonNotebooksWebcast
NumpyNotebooksWebcast
PlottingNotebooksWebcast
UnitsNotebooksWebcast
Tables, I/O and FITSNotebooksWebcast

Coordinates

NotebooksWebcast
ModelingNotebooksWebcast


Table 5. More specialized material

JWST user training
JWST pipeline overview
Webcast starts at 50:00
Astropy & JWST tools overview
Webcast
Image examination (imexam)NotebooksWebcast
Image display toolkit (ginga)Ginga embedded in a Jupiter Notebook

Webcast starts at 26:00
Glue-Ginga Interoperability Webcast 1:03:00 

Photometry (photutils)

Intro Notebooks
PSF-photometry Notebook 

Webcast starts at 55:30
PSF-fitting Webcast 46:00
Data exploration (glue)

Webcast
Customizing Glue Webcast 28:10 

Interactive spectral analysis (SpecViz, MosViz, CubeViz)
Webcast starts at 1:01:30
NIRCam Data Reduction
Webcast
Advanced Astropy TablesNotebookWebcast
Generalized World Coordinate System (gwcs)NotebookWebcast




Published

 

Latest updates