Role of Panel Support Scientists
PSSs represent STScI at the review and provide continuity through their insight into the operations of JWST, STScI and the review process. Their main tasks are to answer questions, to ensure that the panel meetings proceed smoothly, to record the results of the panel discussions, and to bring any critical issues promptly to the attention of SPG and/or relevant STScI technical staff. The PSSs do not take part in the scientific discussions of the panel and do not grade.
The PSS is also present to monitor the panel discussion and ensure that it focuses on the scientific merit of the proposals.The PSS also listens to how the panel discusses the science goals and technical challenges.
If the discussion tends towards comments on the proposing team, their past work, their validity, or their identities, the PSSs job is to refocus that discussion.
The PSS has the authority to stop the discussion on a proposal.
On this page
Before the meeting
- Reviewer assignments: Proposals are assigned to reviewers by SPG based on the information submitted by each panelist. If a panelist identifies a conflict after they receive their assignments, they should contact the PSS and the PSS should re-assign that proposal to another unconflicted reviewer. The PSS may consult with SPG if necessary. The PSS should inform SPG of the re-assignment to ensure that the appropriate information is updated in the database. The Panel Chair should also be informed about reassignments; the PSS should ascertain from the Chair whether they prefer to be updated with each reassignment or to receive the final assignment list after all swaps are made (just before the review).
- Proposal content: The PSS should read (at a minimum) each proposal's abstract and observation summary in order to acquire an overall familiarity with the proposal content of their panel, to assist the Chairs in organizing the material and running the meetings.
- Panelist questions prior to the meeting: PSSs should direct panelists to the JWST Help Desk (https://jwsthelp.stsci.edu) for all questions about science policy or technical performance. Advise panelists to identify themselves as a TAC member when filing a ticket. If the question is about a specific proposal, the proposal ID should be stated in the question.
- Dual Anonymous Issues: Prior to the meeting, panelists are asked to inform their Panel Support Scientist if they have concerns that any proposal is not following the dual anonymous protocols. STScI will determine whether those proposals should be considered non-compliant and flagged for possible rejection. In some cases, there may be an occasional slip-up that can be ignored and does not affect the panel discussion. The STScI Director will make the final decision. If the proposal is deemed non-compliant, it will be removed prior to the panel discussion.
During the meeting
The PSS functions as an executive assistant to the Chair. They organize the materials, provide input on any technical issues noted before the meeting, coordinate real-time consultation with STScI specialists, monitor the grading and final ranking process as well as the status of the proposal feedback comments. More specifically, the principal activities during the meetings are as follows.
- Assist the Chair in the organization of the materials and discussion, and provide any other assistance requested by the Chair.
- Coordinate requests for consultations with STScI experts. A list of contacts and phone numbers for on-call experts will be available in each panel. An STScI specialist should be consulted on every significant feasibility or policy issue, and no proposal may be rejected on feasibility or policy grounds without such consultation. There have been problems on other missions with peer reviewers making their own judgments that later turned out to be wrong. Any consultation must be reported in SPIRIT.
- Panelists should record mandatory comments from the panel: duplications specifically allowed by the panel should be noted in the 'notification comments' of the database as being approved; verified technical issues should be entered into the 'Technical Comments' section of the Web-Reviewer tool.
- The PSS should keep careful notes on any controversies, conflicts, or other exceptional issues, and the resolutions of them recommended by the panels, for subsequent consideration during the Director's Review. These notes should be of a quality such that they would be intelligible to someone else (e.g., SPG staff responding months later to appeals from rejected proposers). An example of such information is keeping track of who leaves the room for each proposal, due to a conflict of interest. Another example would be noting where concerns were raised about the technical feasibility of a proposal, and how those concerns were addressed.
- Although it may be difficult, the PSS should completely refrain from making any scientific, technical, scheduling or policy comments, even if asked to do so by the Chair.
- Monitor progress on the writing of proposal feedback comments (see below) via the Web-based system. Ensure that all proposal feedback comments are finished and signed off by the primary reviewers before they leave the meeting. Ensure that the Panel Chair is aware of the comments status and proof-read the submitted proposal feedback comments as time permits.
- Each panel member gives a numerical grade for each of the 3 grading criteria to the proposal in the web-based review system SPIRIT. We strongly encourage panelists to review proposals as submitted, however, in exceptional cases, the panel may wish to change the time allocation, this should be voted on before grading. The PSS must verify the time allocations are the same as entered in the Web Tool. The importance of care and accuracy in this activity cannot be overemphasized, since it is the only record of the panel's intent.
- After the grading stage follows a ranking stage. Ranks should be modified in the Web-Reviewer tool, as agreed upon by the panel. Grades can not be changed during the ranking stage. The last stage is the discussion of the team expertise for the recommended proposals to the 1N line.
- The final product of each panel will be a ranked list for all proposals, the proposal feedback comments written by the primary reviewer, and a panel report written by the Panel Chair.
Dual Anonymous Issues: Panelists may also raise concerns about proposals during the meeting. In those circumstances, the PSS can help the panel determine the appropriate course of action. If the mistake is slight, the PSS will help re-focus the discussion on the science case. However, if an investigator’s self-revealed identify becomes impossible to ignore, and that identity has a clear impact on the discussion, the proposal should be flagged for disqualification. The PSS should bring this to the attention of the panel if they feel this threshold has been crossed. The STScI Director will make the final decision.
Proposal feedback comments
Before the end of the Panel meeting, the PSS will be asked to proof-read and edit the final version of the proposal feedback comments to make them suitable for dissemination to the proposers. Panel Chairs are also required to proof-read the final version. Use the following guidelines:
- All rejected proposals (those below the Cutoff line), should have a clear statement on why the proposal was "rejected".
- Correct grammatical and typographical errors.
- Eliminate all offensive, gratuitous or personal remarks.
- Remove any explicit identifications of other proposals or proposers in this Cycle under consideration, reviewers, or STScI staff.
- Comments that contain only insubstantial or superficial remarks should be removed, whether for approved or unapproved proposals.
- Remove remarks that do not make sense because the reviewers expected the proposal to be approved, while in the end it was not. However, it is okay if a rejected proposal has positive remarks that are useful; that possibility is covered in the notification letter as a consequence of the heavy over-subscription.
- If any comments mention that the proposal exceeds the page limits, make sure that you verify this personally.
- Make sure you identify any proposals to SPG that violate the Anonymous Review Guidelines.
- Consult with SPG on any questionable cases.
- Inform SPG when a complete set of the final, edited comments are available.