Reforms to CIHR’s Open Suite of Programs and Peer Review Process: Fellowships Pilot – Phase 2: Questions and Answers

What is the Fellowships Pilot – Phase 2?

Fellowships Pilot – Phase 2 is the second phase of the pilot to test aspects of the Open reforms using the Fellowships Program. The first phase (Fellowships pilot – Phase 1) was conducted within the spring 2013 competition and tested Open reforms elements related to the review process only.

Phase 2 will use the fall 2013 Open Fellowships competition to test reforms elements related to both the application and review processes. The Phase 2 pilot includes changes to the current Fellowships structured application process to better align application information with the program’s adjudication criteria. In addition, this pilot will build on the remote review and online discussion functionalities that were implemented in Phase 1 and will introduce a new rating scale and ranking process.

For more information on the reforms to CIHR’s Open Suite of Programs, please see Designing for the Future, The New Open Suite of Programs and Peer Review Process.

Why is CIHR conducting this pilot?

CIHR is committed to ensuring that the transition to the new Open Suite of Programs occurs with minimal disruption. The transition will be a multi-year process that will include a number of pilots which will test the key design elements. Given the scope and scale of the proposed changes, and the intention to learn from the results of the pilots, course corrections and adjustments to design details and timelines may be required along the way.

Why is CIHR conducting this pilot using the Fellowships program rather than a grant program?

Since October 2005, the CIHR Open Fellowships program has used review elements that are similar to those of the new Open Suite of Programs (e.g., structured application, structured evaluation criteria and remote review). By using the Open Fellowships program, CIHR will build upon these elements to allow appropriate testing and validation to inform the new program design with minimal disruption to competition mechanics.

A parallel pilot will test the same design elements within the fall 2013 Knowledge Synthesis Grant competition.

Will the current application process for the Fellowships program change?

Yes. The ResearchNet application form will be simplified and restructured to align with the review criteria.

The application process will be described in detail in the Fellowship Awards – Application Instructions.

How will the current review process for the Fellowships program change?

Building on the first phase of the pilot, we will continue to use an on-line discussion tool and assign three reviewers to each application. In addition, a simplified adjudication scale will be introduced and reviewers will use their ratings to rank applications.

The peer reviewer version of the application will be different. Currently, peer reviewers view a PDF document which displays the application information as it is submitted by the applicant. This requires reviewers to refer to a “Guide to Locating Information” which maps the evaluation criteria to the application content. In the revised process, the peer reviewer version will align applicant information with the adjudication criteria. Each section will have a criterion heading and a brief description of the purpose of each criterion.

Will there be any changes to the adjudication criteria?

No. The adjudication criteria and sub-criteria will not change.

Will this pilot impact the outcome of my Fellowships application?

All applications to the competition will receive the same high quality review and the pilot will not compromise the fairness and transparency of the review process.

Will the results of the pilot be shared with the research community?

Yes. CIHR is committed to sharing the knowledge gained from this and future pilots with the research community.