Introduction 

Data identified with a search may be retrieved in a number of ways, including the MAST Portal download manager,  HTML download, ftp, and with user software. The best method of retrieval is largely a matter of convenience, although requests for very large volumes of data may require special effort. Only authenticated and authorized users may retrieve data within the exclusive access period.


Retrieving JWST data from MAST

Data that have been identified through a search may be retrieved in a number of ways by authorized users. The best method of retrieval is largely a matter of convenience, although requests for very large volumes of data may require special effort.

Anonymous users may access public datasets in MAST without registering and without logging in. But only authenticated and authorized users may access data within the exclusive access period. See the Data Access Policy for details and information on data access restrictions.



MAST portal retrieval options

Users who have identified data of interest within the MAST Data Discovery Portal may retrieve the files (or collections of files) by placing them in the "basket" and clicking the "save" icon. See Portal UI Data Retrieval for details. The MAST Portal also allows for exporting of results data tables and some batch retrieval options including DVD shipment, or staging data for later download. The standard download basket procedure offers the user a choice of zip, tar, tar.gz, or cURL options for the download package, but stipulates a 5.12 GB total file volume maximum.  Above this limit, the user will be obligated to use the cURL script option or the MAST API within a script (see below) for a non-batch retrieval. Batch retrieval options may be ideal if exceeding this volume limit, though this may be true of smaller sizes depending on the user's network connection.

Table 1. Batch retrieval methods

staging
  • Data will be staged to an STScI FTP server.
  • The user will be notified when the transfer is complete and will be given the server location.
DVD
  • The user must provide a full mailing address for shipment.
  • Overnight shipping (where available) once the disk is prepared.
  • DVDs should in most cases be delivered within a week of the request
  • Likely only useful for users with very limited network access
  • Note: The maximum capacity of a DVD (~5 GB) will be too small for larger JWST raw exposures.

Note that a valid email address must be provided for all batch data retrieval requests, though an STScI MyST ID is not required for public data.

MAST portal download examples

Figure 1. A MAST Portal query of M51 for HST data


Figure 2. Selected images are added to the download basket with Minimum Recommended Products still enabled


Figure 3. Download options for the selected images


Figure 4. Batch retrieval options for selected images


Figure 5. Required fields for physical media delivery




The MAST programmatic API

It is possible to retrieve data using scripts in various programming languages, such as Unix shell scripts or Python, through the Mashup API for MAST. See Programmatic Interfaces for details.



Community tools

A variety of community applications can retrieve data from astronomical archives, including MAST. These tools use Virtual Observatory protocols to discover and retrieve data files. Table 2. proviedes with a list of some of them.

Table 2. List of applications and cabilities

ToolCapabilities
AladinSky atlas tool
SAOImage/DS9Image and catalog display tool
TOPCATCatalog joins, subsetting, editing, and visualization
VizieRExtensive VO catalog offerings
VOClientStandalone software for querying and downloading VO data from USVAO

These tools currently lack the ability to provide user authentication credentials to archive services, and so cannot be used to retrieve data archived within the exclusive access period.

VizieR retrieval examples

Figure 6. Searching the hst/obscore catalogue in VizieR for the same obsID as the MAST Portal example


Figure 7. VizieR is able to find the same M51 data found through MAST Portal


Figure 8. The resulting VOTable downloaded for the _drz file








References

MAST Portal user's guide

Programmatic data retrieval

Block has been set to pdf-hide. Contact Shireen if you enter references







Last updated

Published July 11, 2017


 

The format for 'Last updated' is shown below. Enter updates above this internal"Comment" box. Only enter major updates (not typos or formatting changes).


Updated April 5, 2017

  • Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam fermentum vestibulum est. Cras rhoncus. 
  • Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Sed quis tortor. 

Published March 2, 2017