Guidelines for Discussion Panel Chairs

A brief overview of the Review process for Discussion Panel Chairs and Vice Chairs is provided in this article. Overall, the virtual review meetings will proceed in the same way as on-site panel meetings; however, from experience, there are a few things to be aware of and we make some suggestions below.

On this page

Vice-Chairs

Each topical panel has a Vice-Chair. The Vice-Chair is responsible for helping the Chair develop a schedule and assisting the Chair during the virtual meeting, including deputizing when the Chair is conflicted.  We suggest that the Panel Chair works with the Vice-Chair prior to the meeting to develop the overall coordination.

Brief agenda

A very brief agenda for the review meeting is as follows:

  • Introductions of Chair, Vice Chair, Panelists and Panel Support Scientist. Overview of how the meeting will run.
  • Panelists discuss and grade individual proposals in turn; grades are locked after each discussion.
  • STScI makes a ranked list from the grades. Panelists discuss the ranked list, and re-rank, if appropriate.
  • Panelists review team expertise statements for highly-ranked proposals.
  • Panelists discuss (though do not grade) a subset of Large proposals in the panel subject area and provide feedback to the Chair.
  • Panelists should enter the final version of the proposal feedback comments for each proposal for which they are Primary reviewer by the end of the meeting; this may be done at the end or time set aside during the meeting. The Chair or Vice Chair signs off on these once they are complete.



Guidelines and responsibilities

Understanding, patience & kindness

We have panelists spanning many time zones around the world. The meeting times have been truncated to account for this, but we do understand it may be very late or very early for some.

It is also likely that many panelists will be working from home, with partners, children, housemates and pets around, possibly in small spaces. Be understanding about background noise and distractions. That said, we encourage panelists to mute when not speaking to minimize background noise.

Software and hardware

The meetings will be run using Webex. We encourage all panelists to show their videos to allow for face-to-face communication, but recognise this may not be possible in cases where panelists have limited bandwidth. To that end, we encourage panelists to use a wired or stable wireless internet connection, but again recognise that may not be possible in all cases.

Set a schedule, include breaks

Chairs should set a schedule for the meeting and communicate this to the panelists, and PSS ahead of time. We recommend that Chairs schedule blocks of time for different activities (proposal discussions, writing of proposal feedback comments). Don’t forget to include breaks, and stick to them!

When the panel takes a break, please arrange for a note to be projected on the screen with the time that the panel is expected to reconvene. Such a note is helpful for wandering observers to understand the panel's schedule.

Additionally, Chairs should decide and communicate the order in which proposals will be discussed ahead of time so panelists can make sure they are prepared.

It is helpful for everyone if you use an online or fileshare service (such as google spreadsheets) to share your schedule. This way panelists, PSSs, the TAC chair, STScI staff, and observers can see what you plan to discuss and when, and ensures that everyone who needs it always has the most up-to-date version of your schedule.

Ensure everyone has an equal opportunity to contribute

Chairs should set a protocol for the order in which panelists will speak during each discussion and communicate this protocol at the start of the meeting.

Every proposal is assigned a Primary and Secondary reviewer. The Primary starts the discussion by briefly summarising and reviewing the proposal, then the Secondary gives supplementary comments before the discussion is opened up to all panelists. To make sure that everyone gets a chance to speak, and to mitigate the confusion of panelists talking over one another, we recommend that the chairs call on each panelist in turn to explicitly invite their contributions. To ensure that panelists play an equal role in the discussion, and that no one always goes last, we propose that the Chair calls on panelists alphabetically (after the Primary and Secondary reviewers have spoken), starting from the Secondary reviewer.

For example:
Cannon – 7
Herschel – 8
Leavitt (Secondary) – 2
Payne-Gaposchkin – 3
Roman – 4
Rubin – 5
Swope (Primary) – 1
Tinsley – 6

After all panelists have contributed once, the floor will be opened for general discussion. To avoid panelists talking over one another, we encourage panelists to raise their hands to indicate they wish to speak, and the Chair will call on them in turn.

Conflicts of interest

Any panelist who is conflicted on a proposal for any reason must leave the virtual meeting for the duration of the discussion of that proposal (both in the individual proposal discussion phase and the ranking phase if the conflicted proposal is being compared to another). The preferred solution is to add any conflicted panelists to a breakout room in Webex and then recall them to the main meeting once the discussion is over. An alternative solution is for any conflicted panelists to disconnect from the meeting entirely and rejoin once the discussion is over. It is not allowed to turn off audio and video but remain connected to the main meeting.

Ahead of the meeting, panelists should identify the best way to contact them to tell them that the discussion has ended and they can rejoin the meeting. Panelists should also be vigilant during that time and aware that they could be called back at any time. We have set up a Slack channel for each panel and strongly recommend using this, but other methods (email, text, WhatsApp) could be used instead if Slack does not prove to work well. Panelists should make sure that notifications are turned on and/or alert sounds are turned on, as appropriate, for the method they choose.

If the Chair is conflicted on a proposal, the Chair should leave the meeting and the Vice Chair will moderate the meeting. If the Chair and Vice Chair are both conflicted, they will appoint a panelist to handle the discussion of that proposal. If necessary, the TAC Chair can be asked to deputize as Chair; typically SPG will verify that the TAC Chair is available and the PSS will let them know when the proposal is next up for discussion so they can join the panel.

Persons able to join the meeting

The meetings will be private, that is only people with a link may join. There will be a number of people beyond the panelists and panel support who will be given the links and will be able to join.

Each panel is comprised of a Chair and 8 -13 panelists. Each panel will also be assigned a Panel Support Scientist (PSS). The PSS facilitates the process and ensures everything runs smoothly. If there are problems or queries, the PSS can make sure that a member of SPG is aware and can address the issues or answer the questions. In addition, the PSS is present to ensure that the discussions remain focused on the scientific merits of the proposals, and not the proposing teams. Aside from cases of conflict, these people will be in the review meetings permanently.

Additionally, Observers will periodically drop in and out of the virtual meetings to oversee the discussions, just as they would come into and out of meeting rooms during an on-site meeting. Similarly, members of the Science Policies Group will also move between panels as needed. Occasionally, JWST instrument experts will join a meeting if a consultation is required by the panel.

Proposal feedback comments

Panelists are required to provide proposal feedback comments for all proposals for which they are the Primary reviewer, in consultation with the Secondary reviewer – incorporating comments from the Secondary reviewer, the other panelists and from the discussions.

If time permits, we encourage Chairs to set aside time during the meeting to allow panelists to write their proposal feedback comments while other panelists are present to ask for additional feedback. With the truncated timeframe of the meetings, we recognize that this may not be possible.

We encourage panelists to use Slack to message the whole panel or individual panelists to assist in the writing of the proposal feedback comments.

Chairs should check the proposal feedback comments submitted by the panelists (in the case of Discussion panels, the final version submitted by the Primary reviewer)  and ask them to rewrite anything that is not clear or is not professional.

Duplications

The Primary goal of duplication checking is to maximize telescope efficiency by eliminating unnecessary observations. See JWST Duplication Identification and Adjudication Procedures for the TAC

Panel report

The Chair is responsible for writing a brief, high-level summary of the panel deliberations, including anything of particular note that the Chair wants to bring to the attention of the STScI Director (e.g. non-compliance notifications, duplication adjudications, team expertise concerns, science balance, suggested use of unallocated hours, etc). Generally, the panel report should not go into details on any specific proposal, as this is the goal of the proposal feedback comments. 


Resources

  • Example spreadsheet for managing meeting schedule, conflicts, discussion and comments: Schedules.xlsx (note this schedule was made for a 3-day meeting, and we now run 4-day meetings, but the template is still useful)
  • An alternative example spreadsheet with 3 schedule examples, and detailed breakdowns for tracking primaries/secondaries and conflicts: Draft Schedule for STScI TACs (link is to a google sheet)



Next: Webex and Slack Guidelines



Notable updates


Originally published