Cycle 1 Review Selection Criteria and Scoring System

1 Selection Criteria

Proposals reviewed by the topical panels are subject to a two-stage review process: 1) preliminary grading and triage; and 2) the review meeting.

In the preliminary grading and triage phase, each proposal is reviewed and graded by 5 or 6 panel members; those preliminary grades are used to develop a ranked list. Proposals will be assessed on an absolute scale against three primary criteria, with a separate grade given for each:

  • The scientific merit of the program and its contribution to advancement of knowledge – how does the proposed investigation impact our knowledge with the specific sub-field?
  • The program’s impact for astronomy in general – are there broader implications for other science areas and/or insights into larger-scale questions?
  • A demonstration that the unique capabilities of JWST are required to achieve the science goals – suitability for JWST; how much of an advantage does JWST data offer over other facilities? This applies to both GO and AR proposals; Theory proposals should have broad applicability to JWST observational programs.

Reviewers should also consider the strength of the analysis plan in assessing the first two criteria.

Descriptions of additional criteria by type of proposal are given in the Proposal Selection Process section of the Call for Proposals.

2 Scoring System

The scoring should be on an absolute scale with the framework set by the following criteria.

Grade

Impact within the sub-field

Broader impact

Suitability

1

Potential for transformative results

Transformative implications for one or more other sub-fields

Science goals can only be achieved with JWST

2

Potential for major advancement

Major implications for one or more other sub-fields

Major advantages in using JWST over other facilities

3

Potential for moderate advancement

Some implications for one or more other sub-fields

Some advantages in using JWST over other facilities

4

Potential for minor advancement

Minor impacts on other sub-fields

Minor advantages in using JWST over other facilities

5

Limited potential for advancing the field

Little or no impact for other sub-fields

JWST offers little or no advantage over other facilities or the advantages of using JWST are unclear.

Reviewers may submit grades in decimal form, but please limit to one decimal place.

The following examples aim to give guidance in applying these rubrics to grading proposals; reviewers should use their best judgement.

Case 1: Mid-IR observations of gas in young stars

In field

Highly significant improvement in our understanding of gas flow in young stars

1-2

Broader impact

Potential for significant changes in our understanding of gas flows in a wide range of other environments

1-2

Suitability

Mid-IR observations are essential to achieve the science goals and can only be acquired through JWST observations

1


Case 2:  Analysis of archival near-IR imaging of a nearby galaxy for stellar population investigations

In field

Major advance in understanding stellar pops in that galaxy

2

Broader impact

Some implications for stellar pops and stellar evolution in other galaxies

3

Suitability

The increased spatial resolution offered by JWST provides some advantages over other facilities in addressing the science goals. The analysis offers significant improvements and/or additional value with respect to the original use of the data.

2-3


Case 3: Near-IR/Mid-IR spectroscopy of an emission-line galaxy

In field

Moderate increase in understanding of prevalence of star formation in that galaxy

3

Broader impact

Minor implications for the properties of other galactic systems, but no broader impact

4

Suitability

Mid-IR data are not required for the science case; limited gains in performance at near-IR wavelengths as compared with larger ground-based facilities

4-5


Case 4: Developing theoretical tools to characterize dust in Galactic star-forming regions

In field

Potential significant increase in understanding of chemical composition in dusty environments

1-2

Broader impact

Results have broad implications for interpreting dust composition in other galaxies

2

Suitability

The theoretical analysis will enable additional JWST observational programs.

2

The overall grade for each reviewer is the straight average of the three individual grades. The preliminary grade for each proposal is determined by averaging the overall grade from each reviewer. The preliminary grades are used to create a rank order list for each panel and the lowest-ranked proposals (typically 35-40%) are triaged from further discussion.

Grading proposals in the virtual panel discussion

Grading is on the same absolute scale as the preliminary grade, but with an expanded numerical scale to give greater dynamic range in ranking the remaining proposals.

Grade

Impact within the sub-field

Broader impact

Suitability

1

Potential for transformative results

Transformative implications for one or more other sub-fields

Science goals can only be achieved with JWST

3

Potential for major advancement

Major implications for one or more other sub-fields

Major advantages in using JWST over other facilities

5

Potential for moderate advancement

Some implications for one or more other sub-fields

Some advantages in using JWST over other facilities

Reviewers may submit grades in decimal form but please limit to one decimal place.

The overall grade for each reviewer is the straight average of the three individual grades. The preliminary grade for each proposal is determined by averaging the overall grade from each reviewer. Once the grading is complete for all proposals, the rank order list is created. Panels may choose to re-rank (but not re-grade) proposals to adjust the scientific balance.