Triage Process and Discussion List - Cycle 4
This page provides an overview of the triage process and discussion list for the JWST peer review process. Virtual review proceeds via a two-step process; preliminary grades are used to triage the pool of proposals and only the top-ranked of those proceed to the panel meeting.
On this page
Why triage?
Panels typically receive more proposals than can reasonably be discussed during the panel meeting. The triage step removes the lower-ranked proposals from consideration and ensures that the discussion time is focused on the higher-ranked, competitive proposals.
The triage process
During preliminary grading, proposals are read and graded by a subset of the panelists. These grades are averaged together, and the average normalized grades are used to rank the proposals. The spread in grades amongst the panelists is also calculated.
Figure 1. Triage example
Proposals to the left of the green line would move forward to the discussion meeting. Proposals to the right would be triaged.
Triaged proposals with a very high spread in grades are flagged. They are also flagged if, due to conflicts of interest, they were reviewed by a smaller number of panelists compared to the rest of the proposals in the panel.
Reading previously unread proposals that are on the final discussion list
Panelists should be prepared to discuss and grade all proposals on the discussion list during the panel meeting. As they will only have read a subset of proposals for preliminary grading, they should read any additional proposals on the list ahead of the meeting. The deadline for preliminary grades is deliberately set to accommodate this extra reading, and SPG endeavours to turn around the triage as quickly as possible (typically 1-2 days). Timely submission of preliminary grades is critical to allow enough time for extra reading, a panel cannot be triaged until all panelist grades are submitted.
Resurrecting proposals
If a panelist feels strongly that a triaged proposal is worthy of discussion, then they may suggest the proposal for resurrection; panelists may particularly wish to consider proposals with fewer reviewers than average (due to conflicts) or very high grade spread at this stage. Panelists may suggest at most one proposal, and should submit their suggestion to their panel Chair before the start of the panel meeting. This is an optional step and is not required, indeed most panelists do not make suggestions.
When considering whether to suggest a resurrection, bear in mind that not all discussed proposals will be approved for observation, so resurrected proposal would have to climb significantly up the ranking following discussion to be approved for time.
At the start of the panel meeting, suggestions for resurrected proposals will be discussed briefly and a vote taken by the panel on whether or not to resurrect each suggested proposal.
Next: Preparing for the Panel Meeting - Cycle 4