Interpretation Guidelines for the Fall 2013 Knowledge Synthesis Grant Competition Adjudication Criteria

Other format

Table of Contents

Overview

The Fall 2013 Knowledge Synthesis Grant competition has been selected to pilot design elements related to the reforms of the new Open Suite of Programs and peer review process; more specifically those related to the new Project Scheme. The pilot will involve modifications to both the application and review processes of the existing Knowledge Synthesis Grant program.

All applications to the competition will receive the same high quality review and the pilot will not compromise the fairness and transparency of the review process. Applications to the Knowledge Synthesis Grant competition will be structured, and assessed, according to the new proposed Project Scheme’s adjudication criteria. Note that the adjudication criteria herein are intended for this pilot only and may change based on the results of the pilot before they are adopted for the new Project Scheme.

The best knowledge synthesis projects will be selected using a two-stage competition and an application-focused review process. A structured review process will be applied to help focus reviewers’ on specific review criteria at Stage 1 (Figure 1). Stage 2 will involve the integration of the results of Stage 1 review.

Stage 1 focuses on assessing a project’s concept and feasibility, which is founded on a sound and important idea supported by a feasible plan of execution. These criteria are equally weighted in the overall assessment. For the Knowledge Synthesis pilot only, all applications above a pre-determined cut-off will proceed to Stage 2 for selection. (Current thinking suggests that for the new Project Scheme, Stage 2 will focus on the selection “grey zone” applications, which will include applications whose rank orders falls close to the funding cut-off).

Figure 1. Two-stage competition process for the Knowledge Synthesis pilot competition.

Long description: Figure 1

Stage 1 Adjudication Criteria Descriptors and Interpretation Guidelines

Criterion 1: Concept

Sub-criteria to be assessed under Criterion 1 - Concept, include:

1.1 Quality of the Idea

Descriptors

  1. Are the overall goal and objectives of the project well-defined and clear; with distinct outputs that support advances in health-related knowledge, health research, health care, health systems, and/or health outcomes?
  2. Is the rationale of the project idea sound, logical and valid?

Interpretation Guidelines

This sub-criterion is intended to assess the quality of what is being proposed. Is the idea among the best formulated ideas in its field?

  1. The overall goal and objectives of the project should be well-defined and appropriate given the nature and scope of the project.
    • The goal should state the purpose of the project, and what the project is ultimately expected to achieve.
    • The objectives should be action-oriented, highlighting important lines of inquiry and/or activities required to meet the goal.
    • The proposed project outputs (i.e., the anticipated results of the project) should be clearly described and aligned to the objectives.
  1. The project rationale should:
    1. be based on a logical foundation of concepts that is well-supported by evidence (e.g., literature review, previous findings, environmental scan, market analysis, stakeholder input), and
    2. address a valid issue or gap in health-related knowledge, health research, health care, health systems, and/or health outcomes.

Mandatory Requirements (if applicable)

Not applicable.

Other Considerations (if applicable)

Ideas will focus on addressing an issue (e.g., question, problem, need or gap) in any area across the spectrum of health (health-related knowledge, health research, health care, health systems, and/or health outcomes).

Ideas may stem from new, incremental, innovative, and/or high-risk lines of inquiry; new or adapted research and knowledge translation approaches/methodologies; and, opportunities to transfer research findings nationally or internationally. Depending on the nature of the idea, the project may have a research and/or knowledge translation focus.

Projects that have a global health research focus, or include international collaborations, are eligible for support through the Project Scheme. CIHR welcomes all ideas with the potential to improve health outcomes for Canadians and/or the broader global community.

Sample Indicators

Project outputs vary by health area and may include the following examples, listed in alphabetical order:

  • Collaborations or networks
  • Communication and knowledge translation products or resources
  • Competency framework for health professionals
  • Conference proceedings
  • Consultations
  • Databases and evidence repositories
  • Intellectual property claims
  • Invention disclosures
  • License agreements
  • Media interviews (e.g., television, radio, print)
  • Patents (filed or obtained)
  • Policy briefs
  • Presentations at public forums
  • Prevention or intervention programs
  • Products
  • Professional practices
  • Programs or services
  • Public information, resources, and tools
  • Publications, peer-reviewed or other (e.g., theses, journal articles, books, book chapters, workshop reports, synthesis reports, dissemination reports, conference proceedings)
  • Software
  • Spin-off companies
  • Standards and guidelines
  • Theories, models or frameworks
  • Tools, techniques, instruments, procedures or methods
  • Training approaches/curricula

1.2 Importance of the Idea

Descriptors

  1. Is the proposed contribution(s) of the project well-defined, clear and significant with respect to advancing health-related knowledge, health research, health care, health systems and/or health outcomes?

Interpretation Guidelines

This sub-criterion is intended to assess the appropriateness of the proposed contribution(s) and the significance of any advances in health-related knowledge, health research, health care, health systems, and/or health outcomes.

  1. The proposed contribution(s) should be clearly described, and appropriate given the nature and scope of the project. The declared contributions(s) should be realistic, i.e., directly stemming from the project outputs, as opposed to marginally related.

    The proposed contribution(s) should be substantive and relevant in relation to the context of the issue or gap. There should be evidence supporting the significance of the proposed contribution(s).

Mandatory Requirements (if applicable)

Not applicable.

Other Considerations (if applicable)

This criterion is intended to assess the significance of the proposed contributions, and the value of any potential gains, should the issue or gap be successfully addressed. It is not intended to assess feasibility of the project, expertise of the applicant or the potential of success. These will be assessed under Criteria #2: Feasibility.

In cases where projects have a primary implementation, or knowledge translation (application and uptake of research findings) focus, the importance of the idea should be validated as being substantive and relevant by stakeholders, i.e., those who could directly benefit from, or make use of, the project outputs.

Sample Indicators (if applicable)

Significant contributions arising from project outputs can be articulated in a variety of ways, and would vary across the spectrum of health. The following list provides some examples, and is by no means a comprehensive list.

Contributions to the Advancement of Knowledge

  • Important advances in understanding (e.g., informing controversies, examining assumptions, improved theoretical understanding in health professional education, in integrative research, etc.)
  • Significant advances in knowledge (e.g., pathogenesis, cell differentiation, biological pathway, gene-environment interactions, barriers to medication adherence, synthesis)
  • Significant impact on a current area of research (e.g., paradigm shifts)

Contributions to Health Research

  • Development of new areas of research
  • Development of theoretical frameworks/models
  • Development of research methods/techniques
  • Benefit of sustained change in research practice (e.g., ethics considerations)
  • Impact of new research approaches (e.g., multi-disciplinary approaches, horizontal or system-wide research focus, etc.)

Contributions to Health Care

  • Evidence-informed health care practices
  • Evidence-informed health promotion and prevention programs
  • Evidence-informed treatments, therapies, interventions or disease management approaches
  • Evidence-informed health technologies, tools, diagnostics and devices
  • Increased patient satisfaction
  • Reduced burden of illness

Contributions to the Health System

  • Evidence-informed policies (health or other) that positively impact health
  • Efficiencies from improved resource allocations or practices (e.g., cost savings, reduced wait times)
  • Improvements in health care data, information management and infrastructure
  • Reduced health inequalities in a specific population
  • Effectiveness of new or improved health care services, practices and regulations
  • Improved health professional education policies or education system
  • Cultural, organizational or pedagogical changes that have improved health professional competency

Contributions to Health Outcomes

  • Improvements to the health status or quality of life of populations
  • Clinical outcomes of health research interventions
  • Incidence and prevalence of chronic and acute illnesses
  • Changes in culture, attitudes and behaviour that improve health
  • Public health awareness and education (documentaries, classroom/course material, and film)
  • De-bunking stereotypes/myths

Contributions to Economic Opportunities

  • Business opportunities created
  • Creation of jobs from spin-off companies or from expansion of existing company
  • Sales/revenues generated or licensing returns
  • New products, processes or services commercialized
  • License or royalty agreements

Criterion 2: Feasibility

Sub-criteria to be assessed under Criterion 2 – Feasibility, include:

2.1 Approach

Descriptors

  1. Are the approaches and methods appropriate to deliver the proposed output(s) and achieve the proposed contribution(s)?
  2. Are the timelines and related deliverables of the project realistic?
  3. Does the proposal identify potential challenges and appropriate mitigation strategies?

Interpretation Guidelines

This criterion is intended to assess the quality of the Project‘s design and plan, including how and when the project will be completed.

  1. There should be evidence that the research and/or knowledge translation approaches, methods and/or strategies are well-defined and justified in terms of being appropriate to accomplish the objectives of the project.

    Opportunities to maximize contributions should be proactively sought and planned for, but may arise unexpectedly. It is essential that projects have a well-defined knowledge translation approach to maximize the likelihood of achieving the proposed benefits/impacts/positive influence.

  2. Timelines for the project should be appropriate in relation to the proposed project activities. Key milestones and deliverables should be aligned with the objectives of the project, and feasible given the duration of the project.
  3. Critical scientific, technical or organizational challenges should be identified, and followed by realistic plans to tackle these potential risks. An exhaustive list is not expected.

Mandatory Requirements (if applicable)

This section describes mandatory requirements related to Canada’s Health Portfolio Policy on Sex and Gender-Based Analysis.

Projects are expected to integrate gender and sex into their approach, when appropriate, to maximize the utility of project outputs. Any samples, methods or approaches described in the application should appropriately address the consideration of gender and/or sex-based factors, or justify why such consideration is not applicable.

Other Considerations (if applicable)

The intent of the Knowledge Synthesis funding opportunity is to fund the synthesis of existing knowledge; thus, collection of primary data or activities related to primary research are not meant to be supported. In some cases, primary data collection may be warranted, as justified by the applicant(s), to support the synthesis/review (e.g., developing/refining the research question, interpreting the results, and/or validating the interpretation of the results).

All project applications to CIHR should include a knowledge translation approach that is appropriate to the nature of the project outputs. Examples of knowledge translation approaches include: diffusion (e.g., conference presentations, peer and non-peer reviewed publications); dissemination (e.g., tailoring the message and medium to a target audiences); integrated knowledge translation (e.g., involving knowledge users and other stakeholders in the research process); and application (e.g., commercialization, development of evidence-informed health policies or practices). All applications for the Knowledge Synthesis Grant program require an integrated knowledge translation approach.

Definition: CIHR defines a knowledge user as an individual who is likely to be able to use the knowledge generated through research to make informed decisions about health policies, programs and/or practices. A knowledge user can be, but is not limited to, a practitioner, policy-maker, educator, decision-maker, health care administrator, community leader, or an individual in a health charity, patient group, private sector organization, or media outlet.

Sample Indicators (if applicable)

Not applicable.

2.2 Expertise, Experience and Resources

Descriptors

  1. Does the applicant(s) bring the appropriate expertise and experience to lead and deliver the proposed output(s) and achieve the proposed contribution(s)?
  2. Is there an appropriate level of engagement and/or commitment from the applicant(s)?
  3. Is the environment (academic institution and/or other organization) appropriate to enable the conduct and success of the project?

Interpretation Guidelines

This sub-criterion is intended to assess the appropriateness of the complement of expertise, experience and resources among the applicants (the principal applicant(s) and other engaged participants) and their institution/organization, as it relates to the ability to collectively deliver on the objectives of the project.

It is the responsibility of the nominated principal applicant to ensure the proposed project is poised for success.

  1. The applicant(s) should demonstrate the combined expertise and experience needed to execute the project, i.e., deliver the proposed outputs as well as achieve the proposed contribution(s).

    The roles and responsibilities of each applicant should be clearly described and linked to the objectives of the project.

  2. The level of engagement (e.g., time and other commitment) of each applicant should be appropriate to the roles and responsibilities described.
  3. Project applicants should have access to the appropriate infrastructure, facilities, support personnel, equipment, and/or supplies to:
    • carry out their respective roles, and;
    • as a collective, manage and deliver the proposed output(s) and achieve the proposed contribution(s).

Other Considerations (if applicable)

The required complement of expertise will vary by project. Academic and non-academic applicants contribute to the robustness of projects by helping to define/refine research questions, inform the research plan, conduct research, interpret research findings, understand the receptor community, lead dissemination activities, etc. They also bring expertise related to, e.g., appropriateness of the disease model, methodology, theoretical framework, techniques, study population or environment, knowledge translation strategy, market readiness, industry relevance and/or target audiences. Reviewers should assess whether the right experts are engaged to maximize the likelihood of success.

The knowledge user involved in the application should be positioned at the appropriate level of decision making or influence in their organization. Knowledge users may be responsible, and accountable, for the application/uptake of the project outputs.

Access to environments in other organizations may be essential to the conduct of a project. Examples include: hospitals, long-term care facilities, schools, First Nations reserves, industry laboratories, prisons, foreign jurisdictions/locations, community populations, and specialized databases or facilities.

Project environments should be assessed according to their ability to support the proposed project activities. Institutions often function as “networked” environments or interdisciplinary networks, which means there may be multiple satellite environments contributing to the support environment. Reviewers should consider that for smaller institutions, or affiliated research facilities, resources and/or services may be obtained through networks, or may be contracted out.

Letters of support from knowledge user participant(s) and any other key collaborator should highlight roles, capabilities, time and other commitments, and any history of similar work. Knowledge users should also describe how they intend to use the results of the project. Financial commitments (cash or in-kind) are not required of all collaborators, but can be reasonably expected depending on the nature and type of organization, and the anticipated benefits the organization would accrue.

Sample Indicators (if applicable)

Not applicable.

Budget Recommendation

The budget recommendation is intended to inform CIHR funding levels.

While not being factored into the overall rating of an application, CIHR values the experience and perspectives of reviewers in estimating overall budget requirements of the proposed project.

Budget Recommendation

Descriptors

Is the requested funding appropriate to support the project?

Is it realistic and well-justified, taking into account any other sources of funding?

Interpretation Guidelines

Appropriate budget planning should be demonstrated. The requested resources, together with any existing resources, should be adequate to financially support the full scope of the project.

Reviewers are asked to consider the proposed budget of the project, and provide a justified recommendation as to whether the budget should be:

  • Accepted, as described; or
  • Adjusted to the level of $X per annum.

Note that for the Knowledge Synthesis Program, the maximum amount for a knowledge synthesis grant is $100,000 for up to one year, and the maximum amount for a scoping review is $50,000 for up to one year.

Other Considerations (if applicable)

The applicant(s) has been asked to indicate his/her budget using pre-set incremental levels. A precise valuation of all line items in the budget justification is not expected or provided. A detailed item-by-item scrutiny by the reviewer is not expected.

Grant values and durations will be commensurate with the requirements of the project proposed, and will vary depending on the field, proposed approach, and scope of activities.

Sample Indicators (if applicable)

Not applicable.

Supplemental content (right column)