Key Policies for Reviewers
This page describes the Key Policies for those participating in the JWST peer review process.
On this page
Code of Conduct
STScI strives for a collegial and professional environment for all participants in its activities. At any time, please feel free to talk or send a message to any member of STScI staff supporting the review process if you have any concerns. For serious issues or formal complaints, please contact Mercedes López-Morales (AD for Science), Laura Watkins (Head, Science Policy Division), Amaya Moro-Martín (Lead, JWST Science Policy), or Molly Peeples (Cross-mission Policy Scientist).
All participants
All participants in the proposal review process (including STScI staff and observers) are expected to:
- Read the JWST Peer Review Information pages and familiarize themselves with their role and responsibilities.
- Complete their contributions in a timely fashion, or communicate early when this will not be possible.
- Be respectful in any written or verbal communications you have as part of the review process.
- Step in to address abusive or bullying behavior.
- Be respectful of all regardless of differences (professional or otherwise).
- Actively help create an environment free of harassment.
Discussion panelists
In addition, discussion panelists are expected to:
- Be prepared and contribute to the panel review.
- Evaluate the scientific merit of the proposal.
- Be an active participant in the discussions.
- Do not interrupt others or talk over others.
- Keep comments succinct and to the point and thus give everyone the opportunity to contribute to the discussion.
- Be polite and professional in your written feedback comments, especially when providing critical comments.
- Minimize periods of unavailability to avoid disruption for the other participants.
Panel Chairs and Vice Chairs
In addition to the code of conduct for discussion panelists, Chairs and Vice Chairs are expected to:
- Lead by example in creating the appropriate environment for free and professional discussion.
- Lead the panel in a collegial and welcoming way and respond immediately to any abusive, bullying or unprofessional behavior.
- Proactively encourage participation of reviewers who may be less experienced at panel reviews.
- Proactively solicit input from each panel member in the discussion of each proposal; ensure that the discussion is not dominated by a few reviewers.
- Keep the discussion moving and end on time to allow for sufficient time and discussion for all the proposals in the panel.
- Keep the discussion focused on the strengths and weaknesses of the proposal, and no other tangential topics.
External panelists
In addition, external panelists are expected to:
- Evaluate the scientific merit of the proposal.
- Be polite and professional in your written feedback comments, especially when providing critical comments.
Confidentiality
- Proposals and review documents should not be distributed or used in any manner not directly related to the review process.
- Aspects of the review may not be discussed with anyone other than those STScI staff participating in the review activities.
- You should not discuss the outcome of the panel evaluations – now, or in the future.
- Confidentiality carries from prior years: Do not discuss or compare prior years' proposals in this review, even with panel members who also served in prior years.
- If a video-conference panelist, make sure nobody besides yourself can hear the panel discussion.
- Individual reviews should be independent; do not consult with other panelists before the panel convenes.
- Please purge all review files from your computer after the review.
Your participation in the review is confidential until after the results have been announced.
- Do not post comments to social media (Facebook, X, Instagram, TikTok, etc.) regarding the content or your participation in the panel meeting.
- A summary of the TAC results will be published in the STScI Newsletter; only then should you feel free to update your C.V., etc. to include your participation in the TAC.
Conflicts of Interest
Why are conflicts of interest important? Our goal is informed, unbiased discussion of each proposal's science case as described in the Selection Criteria and Scoring System section:
- Grading committee members should have neither direct nor indirect interest vested in the outcome of the review.
- The subset of the review committee discussing the proposal should have sufficient knowledge to assess the science.
It is critically important that conflicts of interest, or the appearance thereof, are avoided during the selection process.
What is considered a conflict of interest?
The following are identified as conflicts:
Personal involvement (PI, co-PI, or co-I, i.e., an investigator)
Recent (within 5 years) former advisor/student of an investigator
Involvement in a closely competing proposal regardless of whether that proposal is also before your panel.
Proposals are judged to be "closely competing" if their scientific goals or their observations of the same targets are sufficiently similar that an impartial panel, knowing one had been approved, would be unlikely to approve the other. Such conflicts should be direct, specific, and determined on a case-by-case basis; panelists or chairs should not be excluded from considering an entire sub-category of proposals (i.e., they cannot declare to be conflicted on an entire sub-category of proposals).
- Strong competitor of an investigator on the proposal
Close personal ties (family, etc.) with an investigator
Close collaborator of an investigator on the proposal
- A close collaborator is someone who is an active collaborator on a current research program, an active collaborator on at least 3 projects completed within the last 3 years, or an active co-author on 3 or more papers in the last 3 years. Please only list collaborators with whom you have worked directly on multiple projects; do not list every co-author or colleague from large-scale projects. Co-membership of a large consortium, such as SDSS or a JWST ERS team, is not a disqualifying factor.
- Any other reason why a reviewer feels they might be unable to provide an unbiased review (positive or negative).
How can I report a conflict of interest?
Reviewers are asked to identify potential conflicts of interest as part of the recruitment process. Since proposals are anonymized, STScI staff will identify conflicts using the conflict lists supplied by the panelists.
When reviewers receive their assigned proposals, they should check them all for additional conflicts as soon as possible.
If a reviewer feels that they have a conflict with any additional proposals, they should contact their Panel Support Scientist (PSS) immediately and not take any further action with that proposal. They should never discuss the reason of the conflict with anyone in the panel. Conflicts may be reported at any time during the review process, but the sooner the better. We will reassign conflicted proposals, so it helps everyone if we know about these early.
What happens in case of a conflict of interest?
Conflicted panelists must not read, provide grades for, or comment on a conflicted proposal. Proposals on which a reviewer in conflicted will not be visible to them in SPIRIT.
During the discussion meeting, conflicted panelists will be asked to leave the room/call for the discussion of that proposal and only be allowed to reenter once the discussion is over. During the ranking phase, conflicted panelists will be asked to leave the room/call if the proposal is discussed or compared to another. There should be no discussions/speculations on the nature of the conflict.
Discussion panelists should take care not to discuss specifics or comments from proposals in Slack where conflicted panelists might see the discussion.
Dual Anonymous Peer Review (DAPR)
More information about Dual Anonymous Peer Review can be found on the JWST Dual Anonymous Peer Review page in the Call for Proposals.
Why dual anonymous peer review?
The primary objective of the review is to select the best-justified science.
Removing the names of the proposal team from the proposal allows reviewers to focus directly on the science, rather than the scientist. The TAC panels and chairs rank proposals in order of scientific merit and recommend the resources that should be allocated to each. The experience of the team is not a consideration for the science ranking.
Reviewer guidelines
Do not spend time attempting to identify the team or the principal investigator.
A reviewer's preliminary grading should center on the scientific merit within the field, the broader importance for astronomy, and JWST’s unique capabilities must also be required to achieve the scientific goals. The discussion during the meeting should focus on the specific criteria. Chairs and PSSs should be quick to refocus or terminate discussion when it moves to PI or team.
Please note that the goal in anonymizing proposals is not to reduce content to the extent that useful information is eliminated. Proposers are encouraged to present relevant past observations and theoretical analyses, together with complementary observing programs. However, as detailed in the Call for Proposals, the information should be presented in an anonymized fashion. Any privileged information should be cited as having been obtained through private communication.
The (non-conflicted) SPD staff will review the text and make a recommendation to the Director. Proposals with very minor infringements may be allowed to proceed, others will be flagged for disqualification. Appropriate feedback will be provided to the proposers after the selections are announced.
Team expertise review
Observing time with JWST is openly available to any scientist who presents a highly compelling scientific case. However, that observing time is also a highly valued resource that must be used responsibly. To that end, the panels will have the opportunity to review a proposing team's expertise to verify that they are capable of executing the program they have described, and in disseminating those results. This review will take place after the panel has ranked all programs based on the selection criteria. Panelists will be given the opportunity to review prepared statements by the proposers on their team's expertise and background. If, by consensus, the panel has serious concerns that a team lacks sufficient expertise or resources to execute the project described in the proposal, they may flag that proposal for potential disqualification to the Director. Panels must provide an appropriate justification for that action. This process does not change the ranking of other proposals. If a panel does flag a proposal for disqualification owing to lack of necessary expertise, it is not replaced by a lower-ranked proposal.
Use of Generative Artificial Intelligence (GAI)
Reviewers are forbidden from using Generative Artificial Intelligence (GAI) tools to assist with any part of their reviews since this violates the confidentiality of the review process. This means that it is forbidden to upload proposal contents or review materials to GAI tools. It is also forbidden to upload your written comments to GAI tools for the purpose of constructing your final review or polishing the language.
Next: Selection Criteria and Scoring System