STScI has implemented a dual-anonymous proposal review process, where the identities of the proposing team are concealed from reviewers. Dual anonymous reduces bias in the review process by focusing on the scientific merit of the proposal rather than the participants.

STScI has a responsibility to simultaneously ensure that the community has equal opportunity for the use of JWST and that the best science is being done with the finite amount of observing time available. The Institute places a high value on the equity and integrity of the proposal review process.

The focus of the TAC review is to recommend the best science. The identity of the proposing team should not be a consideration in making this judgement. However, analysing data from many cycles, we noted that there were systematic demographic differences in proposal success that suggested that unconscious bias might be playing a role in the TAC deliberations. Several studies have also shown that a reviewer's attitude toward a submission may be affected, even unconsciously, by the identity of the lead author or principal investigator. Independent studies of our reviews suggested that a double-anonymous process might help resolve this inequity, and may balance out other areas of potential bias.

In the spring of 2018, STScI convened a working group from the astronomy community to explore the idea of a dual-anonymous system and issue a set of recommendations to the STScI Director. The working group's report, along with detailed instructions to proposers and reviewers, and a list of FAQs, can be found on the Working Group's website. The dual-anonymous system was successfully implemented during the Delta 26 proposal review, has been successfully used in every HST and JWST proposal review since, and will be continued this Cycle. The goal of Dual Anonymous Peer Review is to enable each reviewer to focus on the science, not the proposing team. A summary of the dual-anonymous process guidelines, along with a description of how the review process works, is given below.

The Dual Anonymous review process

As in past cycles, proposers submit their proposals through APT. However, the PDF attachment that is uploaded containing the scientific and technical justifications must be anonymized following the guidelines below. Additionally, proposers must submit, via the Astronomer's Proposal Tool, a separate section titled "Team Expertise and Background." The review panels (and the Executive Committee) will conduct their review without seeing any of the names associated with the proposal, and without seeing the information in the "Team Expertise and Background" section. The panels will discuss the proposals and generate a final ranked list of proposals that are recommended for selection. In addition to the Panel Chair, each review panel (including the Executive Committee) will have a full-time "Leveler" present in the room during all panel discussions. The job of the Leveler is to ensure that discussions remain focused on the scientific merit of the proposal.

Once the ranked list is set, the panels will be given access to the "Team Expertise and Background" information associated with each proposal recommended for implementation. At this point, proposals may only be flagged for downgrade, where a downgrade would result in a non-selection of the proposal. If a proposal is downgraded after the team expertise review, other lower ranked proposals may not be upgraded to take its place. This flag, assigned by majority vote of the panel, should only be used in the most extreme circumstances of a team being clearly unqualified to undertake the work proposed. Should a proposal be suggested for downgrade, both the Panel Chair and the Leveler will participate in the discussion about why this recommendation is necessary. A detailed description of the reason for the flag must be given. This flag will then be passed on to the STScI Director, along with the proposal's initial ranking, and a statement by the panel on the rationale for flagging the proposal. The Director will make the final decision, in consultation with appropriate personnel from STScI, including the Science Mission Office (SMO), JWST Mission Office, ESA Office, and operations/scheduling staff. Finally, any proposals that are downgraded will have the reasons for downgrade passed on to the proposers. The same process will be applied to Large proposals by the Executive Committee.

Guidelines for the PDF submission

Provided here are guidelines to assist proposers in preparing their proposals, specifically their PDF Submissions, to help conceal the identities of the proposers, and ensure a fairer proposal evaluation process. The anonymous review does not mean proposals will be accepted from anonymous sources. As with previous cycles, proposers must still enter the names and affiliations of all investigators into the APT system. APT will not include names or affiliations in the versions generated for the reviews.

While APT will largely obscure the proposing teams identities in cover materials, it will not change or alter information contained in the PDF submission. Thus, it is necessary for proposers to take additional steps to further anonymize their PDF attachment before it is uploaded to APT. Below are some guidelines to accomplish this:

• Do not include author names or affiliations anywhere in the PDF attachment. This includes, but is not limited to, page headers, footers, diagrams, figures, or watermarks. This does not include references to past work, which should be included whenever relevant (see below).
• Referencing is an essential part of demonstrating knowledge of the field and progress. When citing references within the proposal, use third person neutral wording. This especially applies to self-referencing. For example, replace phrases like “as we have shown in our previous work (Doe et al. 2010)” with “as Doe et al. (2010) showed...” Do not refer to previous campaigns using JWST or other observatories in an identifying fashion. For instance, rather than write "we observed another cluster, similar to the one we are proposing under JWST program #XXXXX," instead write "JWST program #XXXXX has observed this target in the past..."
• We encourage references to published work, including work citable by a DOI, but do not claim ownership. In general, only use the first person possessive when talking about future work by the proposal team.
• It may be important to cite exclusive access datasets, ancillary data from private facilities or non-public software that may reveal (or strongly imply) the investigators on the proposal. Please include those references if they are germane to the proposed science, but without claiming ownership. We suggest proposers use language like "obtained in private communication" or "from private consultation" when referring to such potentially revealing data or facility access. Reviewers are instructed to accept such statements without requiring more justification in the proposal text, although that can be included in the Team Expertise section.
• Do not include acknowledgements, or the source of any grant funding. 
• The goal of dual-anonymous peer review is to remove the focus of the proposal from the proposing team and place it on the proposed science. Thus, discussions of the team's experience or composition is strongly discouraged, even if done so in an anonymous fashion.

It takes effort by authors to anonymize their PDF submissions. Some examples of re-worked text can be found in Example text for anonymous proposing. Please take sufficient time to prepare the manuscript, especially if planning to resubmit a proposal from an earlier cycle or other submissions. 

Anonymizing a proposals is not an excuse to omit relevant scientific information. Proposers should describe the past work in the field, and how this proposal will improve, build-upon, or complete that past work. Many successful proposals include a discussion of stated-sample goals or statistical completeness and how the proposed work fits into this broader context. Similarly, proposals may also discuss the uniqueness of the sample, and goals in comparison to similar work.

Team expertise and background section

As part of the proposal submission, proposers should complete the "Team Expertise and Background" section in APT. This section should provide a brief description of the expertise, background, and roles of key team members, as they relate to the science proposed. This section should be limited in length; for most proposals, a paragraph or two will suffice. For proposals with a large number of Co-Investigators, it is not necessary to report on the qualifications of every team member, nor is it necessary to provide a bio of all team members. If proposers wish, they can identify the PI in this section. An example is provided in the Proposer Guidelines in Anonymous Reviews. 

Please note: the text box will support ascii text. Special text markup and LaTex are not supported.

Compliance

Proposals must be anonymized in accordance with the guidelines above. Compliance with this policy is mandatory. Proposals received with violations will be subject to disqualification before the review-panel stage. Proposals with very minor infringements may be allowed to proceed under exceptional circumstances. Feedback will be provided to proposers regarding any violations.

A possible concern that may arise is the following: "I've made every effort to anonymize my proposal, have followed all the guidelines, changed all my references to third-person, but I fear that my work is so specialized (or my analysis methods so unique) that panelists who know me will still be able to figure out who I am. Will my proposal be disqualified?" So long as the guidelines above are followed, the answer is NO, such a proposal will not be considered to be in violation. It is not necessary to "water down" or obscure your science, your methods, or your tools; it is simply your responsibility to write about them in the third-person, in a way that does not intentionally identify yourself. 

Self-plagiarism is not acceptable. Not only is it unethical, it goes against the spirit of Dual Anonymous review by identifying authors of one proposal as authors of another, even if not by name. Examples of self-plagiarism include, but are not limited to, using identical portions of text in multiple proposals submitted in the same cycle, and submitting a proposal identical to one approved in a previous cycle (resubmission of unsuccessful proposals from previous cycles is acceptable). Some re-use of text from confidential sections of a proposal may be allowable. Instances of self-plagiarism risk being disqualified from the review.

How your anonymous proposal is reviewed

Proposers need to write a proposal that concentrates on the science and is properly anonymous in regard to the Proposal Team, but the reviewers also have responsibilities to follow the dual-anonymous process, detailed in Dual Anonymous Proposals Guide for Reviewers. The primary objective of these reviews is to select the best science, not the best science teams. Panels, facilitated by Panel Chairs, rank proposals in order of scientific merit, and recommend the resources that should be allocated to each. The experience of the team with JWST or otherwise is not a consideration until after rankings occur. Reviewers are instructed to not spend time attempting to identify the team or the principal investigator. All accepted proposals are assigned a Program Coordinator who works with the PI to finalize the Phase II submission for feasible observations. MAST provides "science ready" data for most uses, and there is help/documentation for further data processing. A reviewer's preliminary grading should be centered on the main review criteria. This includes technical issues in the design of the study, as described in the Description of Observations section and elsewhere. The discussion should focus on the scientific merit of the proposal. Chairs and Levelers are instructed to refocus or terminate discussion when it moves to PI or team. The guidelines given to reviewers can be found in the Dual Anonymous Proposals Guide for Reviewers. 

Next: JWST Proposal Submission Policies