Final Ranking

A description of the final ranking process for discussion panels is provided in this page.

On this page

Number of hours and goals of ranking

Each panel has a nominal allocation of N hours, which will be communicated by SPG. The number of hours is different for each panel and the allocations are determined by the relative proposal pressure and hour pressure across the panels. Panel members should review the rank order list to determine whether the highly-ranked proposals above the nominal cutoff line ("the 1N line") provide an appropriate science balance for the panel. There may be a consensus that some science areas have been unduly favored. There may also be cases where the chair identifies highly ranked proposals that have a science overlap with proposals highly ranked by another panel. The panel members can make a consensus decision to re-rank (but not re-grade) proposals to provide an appropriate reflection of the science topics reviewed by the panel.



Re-ranking process

Whenever two proposals are being discussed together, panelists conflicted on either proposal may not be present for the discussion.

In re-ranking proposals, panels may directly compare proposals, irrespective of their relative ranking, that are judged to have very similar science to the extent that the panel may recommend executing only one proposal. Panelists conflicted with either proposal may not vote on the re-ranking. If the panels choose to do only one proposal, the other proposal is moved to the 2N line and the Proposal Feedback Comments for that proposal are adjusted to reflect the discussion.

In all other cases, panels may only compare and vote on adjacent proposals. This is to minimize conflicts. Thus, if a panelist advocates raising a proposal in position 14, it must be compared and voted against proposal 13 - panelists with conflicts on #14 and #13 may not vote. If it is raised to position 13, it can be compared against proposal #12 – again, panelists conflicted on either proposal may not vote. And so forth until the ranking is established. Note that the exact ranking is most important close to the 1N line for each panel.



Medium proposals

Each panel is allocated a separate pool of hours for Medium proposals based on their Medium hour pressure. Only Medium proposals above the 1N line are eligible for this allotment. Any hours from the Medium pool that are not allocated may not be used for Small proposals. Panels should not inflate the rank of a Medium proposal in the case where they would not naturally use up this allocation.

If a panel has a Medium proposal above the 1N line that can only partially be filled by the Medium hour allocation, remaining hours may be allocated from the panels allocation of hours for Small proposals, but this is not required. Similarly, the panel can keep additional Medium proposals in their ranked list to be filled by Small allocation hours, but this is not required. Any Mediums removed from the ranked list should have their ranks adjusted by pairwise comparison with other proposals; however, in making those comparisons the panelists are free to use valid considerations beyond the individual merits of each proposal, such as ensuring a scientific balance of the approved program. The only exception to this is if they duplicate science in which case they can be directly removed without any further pairwise comparison.



Final recommendation

Panelists are asked to rank proposals all the way down to twice the hour allocation (the 2N line); this is in case any changes need to be made to the top-ranked proposals (for example, an approved proposal in another panel proposes the same observations, thus making the proposal in the panel a duplication) or if it's necessary to add additional programs at a late stage in the cycle. Panelists are also asked to set a do-not-approve line, if they deem it appropriate.



Next: Large, Treasury and Legacy Proposals


Latest updates


Originally published