JWST Data Formats

Various JWST data products are produced in one of several standardized data formats, as described in this article. 

On this page

See also: Understanding JWST Data Files

Working with JWST data requires an understanding of JWST file types and steps in the JWST Science Calibration Pipeline that produce them.

JWST file formats

JWST data produced by the JWST Science Calibration Pipeline can include Flexible Image Transport System (FITS) files, Advanced Scientific Data Format (ASDF) files, JavaScript Object Notation (JSON) files, and Enhanced Character Separated Values (ECSV) files. In general, all the science data include FITS and ASDF extensions, but these formats are also used for calibration reference files.  

  • FITS format extensions, on top of the science pixel values, provide relevant information about the science data pixels, e.g., uncertainty or variance estimates for each pixel, data quality flags for each pixel, integration times for each exposure, metadata for data groups or for data that was combined into a single product.
  • ASDF format extensions are used to provide more complex information about that data, like data model metadata or world coordinate system information.
  • ECSV files are products generated in the last stages of calibration and for certain observing modes. These contain a catalog of sources or specific photometric information derived by the science calibration pipeline.
  • Finally, JSON files contain information regarding the way the science data should be associated by the calibration software and according to the rules for each of the observing templates defined by the different instrument teams.

Once JWST data are obtained, telemetry data are received by the Data Management System (DMS) and processed to extract the science data. The FITS header will contain keywords required by the FITS standard and keywords relevant to the observation that are extracted from the telemetry packet headers and science image headers.

Multi-extension FITS format

Flexible Image Transport System (FITS) is a standard format for exchanging astronomical data, independent of the hardware platform and software environment. FITS format files consist of a series of header data units (HDUs), each containing 2 components: an ASCII text header and binary data. The header contains a series of keywords that describe the data in a particular HDU; the data component may immediately follow the header.

For JWST FITS data, the first HDU, or primary header, only contains header information in the form of keyword records with an empty data array, which is indicated by the occurrence of NAXIS = 0 in the primary header. The primary header may be followed by one or more HDUs called extensions, which may take the form of images, binary tables, ASDF files, or ASCII text tables. The data type for each extension is recorded in the XTENSION header keyword.

ASDF format

The Advanced Scientific Data Format (ASDF) is a next generation, human-readable, hierarchical metadata structure made up of basic dynamic data types such as strings, numbers, lists, and mappings. Data is saved as binary arrays. It is primarily intended as an interchange format for delivering information about the science instruments or how products were created to scientists or, for example, between stages of the JWST Science Calibration Pipeline. ASDF files are added to certain calibration pipeline products and are part of the  reference data used by the software. As an example, distortion and spectral models needed to transform detector positions to a world coordinate frame are in ASDF format. 

ECSV format

The Enhanced Character Separated Values (ECSV) format, which is standard for the interchange of tabular data in a text-only format, is used to store a catalog of derived data for sources identified in some steps of the calibration pipeline. This file includes a header section with the definition, data type, and description for the columns, and a data section with as many rows as sources identified in an image. Besides a simple comma-separated delimited text file reader, these files can also be read, modified, or created using the Astropy ECSV code.

JSON format

The JavaScript Object Notation (JSON) is a language-independent data format that many modern programming languages understand, and is used to transfer populated data structures. The calibration pipeline uses this type of file to provide information about relationships between multiple exposures or associations. This is also used by some calibration reference files.

Latest updates

Added information about JSON files

Originally published