Appendix 4: Instruments

Midterm Evaluation of the Pandemic Preparedness Strategic Research Initiative

[ Table of contents ]

Key informant interview guide, v4 (19.12.08)

Interviewee: _______________________________________________

Interviewer: _______________________________________________

Date: _______________________________________________

Thank you for agreeing to participate in this evaluation of the Pandemic Preparedness Strategic Research Initiative. Its objective is to assess the overall design and implementation of the Initiative, in order inform future developments in the PPSRI, as well as similar strategic initiatives to be undertaken by III.

Note that interviewees will be asked to respond to only those questions for which they judge they can contribute an informed opinion.

1. Introduction

1.1 To begin, could you please describe how familiar are you with the Initiative? What has been your involvement with the PPSRI?

2. Effectiveness of priority setting

2.1 What is your overall opinion on the research priority setting processes implemented under the PPSRI? Which aspects seemed to work well? Which aspects seemed to not work so well?

Follow up if not addressed in response provided above - To what extent were the PPSRI research priority setting processes appropriate?

Follow up if not addressed in response provided above - How would you assess the processes in terms of:

• Timeliness?
• The consultation mechanisms used?
• Who was and was not involved?

a) What alternatives do you consider might have been more appropriate? What changes would make improvements?

2.2 What is your overall opinion on the actual set of priorities that have been selected for Canadian research in pandemic preparedness?

Follow up if not addressed in response provided above - To what extent did the process arrive at the most appropriate set of priorities for Canadian research in pandemic preparedness?

a) When you consider the actual set of priorities selected, is there any national and/or international duplication of effort with this set of priorities?

• If yes, in which areas? What changes to the process of priority setting would have assisted in avoiding duplication?
• If no, how was this avoided?

b) When you consider the actual set of priorities selected, is there sufficient room for complementarity with other initiatives?

• If yes, how was this achieved? With which other initiatives?
• If no, what changes to the process of priority setting would have assisted in ensuring or enhancing complimentarity with other initiatives?

2.3 Were you involved with the CIHR Pandemic Preparedness Task Group? How effective was it as a priority setting mechanism for the PPSRI?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

• How necessary it was to convene such a group
• Task Group's role
• Membership

a) What alternatives do you consider might have been more appropriate or effective? What changes would make improvements?

3. Effectiveness of partnership development

3.1 To what extent has the PPSRI had an influence on national and international partnerships? (Describe observed impacts)

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

• Which partnerships have been most and least successful?
• Partnerships that should be developed but have not been - which ones, and why?
• What alternatives do you consider might have been more successful?

3.2 As far as can be seen at this point, what have been the impacts of partnerships on:

• coordination and integration of national and international research programming?
• resource leveraging?
• research duplication and complementarity?

a) What alternatives do you consider might have generated more impact? What changes would make improvements?

3.3 What are your opinions and observations on the outcome(s) that the partnership structure under the PPSRI has had on ongoing initiatives of partners? On the PPSRI itself? (Describe observed outcomes)

a) What alternative partnership structures do you consider might have been more effective? What changes would make improvements?

3.4 What are your opinions and observations on the outcomes(s) that the reporting strategy of the PPSRI has had on ongoing initiatives of partners? On the PPSRI itself? (Describe observed outcomes)

a) What alternative reporting strategies do you consider might have been more effective? What changes would make improvements?

4. Appropriateness of program design

4.1 What is your perspective on whether or not the set of funding opportunities offered is assisting the PPSRI in achievement of the Initiative's objectives?

Follow up if not addressed in response provided above -

• Which of the funding opportunities do you consider most and least successful?
• Are there gaps in the funding opportunities made available so far? If so, which?
• Is the overall approach of strategic competition effective in ensuring the most promising research is funded? What would have been alternatives? How would they have been more effective?
• Is this the most effective approach to ensure both short and long term capacity development? What would have been alternatives? How would they have been more effective?

a) What alternative funding opportunities do you consider might have been more appropriate? What changes would make improvements?

4.2 What are your main observations on the communication strategy used to launch the funding opportunities?

Follow up if not addressed in response provided above -
Which aspects of the communications do you consider most and least successful?

a) What alternatives do you consider might have been more effective with respect to the communication strategy? What changes would make improvements?

5. Achievement of desired funding opportunity outputs

5.1 What is your perspective on the uptake of the funding programs?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

• Whether or not the relevant research communities responded to the RFAs
• Have there been gaps in response? If so, which areas? Which teams? Which key players?
• The quality of submissions (e.g., teams, proposed projects)

a) What alternatives do you consider might have generated more satisfactory uptake? What changes would make improvements?

5.2 What are your thoughts with respect to the level of coverage across the intended fields of research priorities?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

• Which priorities are more and less well represented across funded projects
  • Vaccines and immunization
  • Virus biology and diagnostics
  • Prevention and treatment
  • Ethics, legal and social contract

a) What alternatives do you consider might have generated more satisfactory coverage? What changes would make improvements?

5.3 What is your opinion with respect to whether or not the funded projects will result in research training in areas related to pandemic preparedness?

Follow up if not addressed in response provided above -
Would these training opportunities likely have existed without PPSRI funding?

a) What alternatives do you consider might have generated more training opportunities?

5.4 What is your opinion on the likelihood that there will be sufficient knowledge translation of the results from the funded projects?

Follow up if not addressed in response provided above -
Do project proposals include realistic, relevant plans for knowledge translation?
Are there the appropriate end-users involved in project teams for this to occur?

a) What alternatives do you consider might ensure appropriate knowledge translation occurs for projects?

6. Success of PPSRI networking activities

6.1 From your observations, what impact has the PPSRI had on the facilitation of communication and networking among researchers involved in pandemic preparedness research?

Follow up if not addressed in response provided above -

• How has the PPSRI facilitated communication and networking? (examples)
• What are the more formal vs. more informal opportunities provided?
• Have all relevant teams and individuals been provided with networking opportunities?
• How satisfactory has been the uptake of these opportunities?

a) What alternatives do you consider might have been more successful? What changes would make improvements?

6.2 Have you observed whether or not the communication and networking activities had any impacts on enhanced collaboration? Increased research capacity?

Follow up if not addressed in response provided above -

• If yes, how was this achieved? Are these more formal or informal activities?
• If no, what changes to these activities would have assisted in ensuring or enhancing collaboration? Increasing research capacity?

7. Final comments

7.1 Do you have any other comments on the PPSRI and how it, for future strategic initiatives, might be improved?

Thank you for your collaboration!

E-survey of researchers

Cover E-mail

Thank you for your interest in the evaluation of the Pandemic Preparedness Strategic Research Initiative (PPSRI).

Purpose: This survey is part of the midterm evaluation of the PPSRI, being conducted by the Institute of Infection and Immunity (III) of the Canadian Institutes of Health Research (CIHR). The evaluation will assess the overall design and implementation of the PPSRI, in order to inform future developments in this initiative, as well as similar strategic initiatives to be undertaken by III.

Deadline: Please complete this survey by February 28, 2009.

Questions: If you have any questions about the evaluation or the survey, please do not hesitate to contact Celine Pinsent, Project Manager, at 613-230-5577 X 226 or cpinsent@ggi.ca, or Carol Richardson, at 519-661-3228 or carol.richardson@schulich.uwo.ca.

Web-based Survey

A. Awareness

1.	I have heard of the Pandemic Preparedness Strategic Research Initiative.
	Yes No Don't know
2.	My research work is relevant to pandemic preparedness and influenza research in the following areas:
	- Vaccines and immunization
	Yes No Don't know
	- Virus biology and diagnostics
	Yes No Don't know
	- Prevention and treatment
	Yes No Don't know
	- Ethical, legal or social aspects
	Yes No Don't know
	- Other area related to pandemic preparedness and influenza research:
	Yes No Don't know
	If yes to above: Please specify which other area:

B. Decision to apply
Applicants to the PPSRI funding opportunities

3. Were any of the following a factor in your decision to submit your proposal to the PPSRI?

Yes	No	Degree of fit of my research with the themes
Yes	No	Qualifications of me and my co-applicants
Yes	No	Size of the grants offered
Yes	No	Type of grants offered (e.g., team grants)
Yes	No	Duration of the grants offered
Yes	No	Relative prestige compared to other programs available
Yes	No	Program seemed to be targeting specific groups of researchers
Yes	No	Expected involvement of students in the project
Yes	No	Needed more research funding
Yes	No	Timing of RFA was good
Yes	No	Past experience with applications to CIHR
Yes	No	Thought my/our chances of success were good
Yes	No	Other reasons: ______________________________________

Non-applicants to the PPSRI funding opportunities

3. Were any of the following a factor in your decision NOT to submit a proposal to the PPSRI?

Yes	No	Was not aware it existed
Yes	No	Degree of fit of my research with the themes
Yes	No	Qualifications of me and my co-applicants
Yes	No	Size of the grants offered
Yes	No	Type of grants offered (e.g., team grants)
Yes	No	Duration of the grants offered
Yes	No	Relative prestige compared to other programs available
Yes	No	Program seemed to be targeting specific groups of researchers
Yes	No	Expected involvement of students in the project
Yes	No	Already had enough research funding
Yes	No	Timing of RFA was poor
Yes	No	Past experience with applications to CIHR
Yes	No	Thought my/our chances of success were not good enough
Yes	No	Other reasons: ______________________________________

All successful and unsuccessful applicants to the PPSRI funding opportunities
C. Views of the design and competition process

To what extent do you agree or disagree with the following statements?

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don't Know
4. I felt there was conflict of interest in the way the PPSRI was designed.
5. The PPSRI's competitions' peer review committees had adequate expertise.
6. I was aware of the criteria used to judge the PPSRI funding applications.
7. The PPSRI competitions' funding decisions were fair.
8. I was provided with sufficient information with respect to the reason(s) my PPSRI funding application was successful or unsuccessful.
9. I felt there was conflict of interest in the way the PPSRI applications were reviewed.

Successful applicants to the ppsri funding opportunities
D. Communication, networking, impacts on research and knowledge transfer

Please note: if you have received more than one grant through the PPSRI, please answer in terms of the most recent.
To what extent do you agree or disagree with the following statements?

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don't Know
10. Communication about the PPSRI has been adequate.
11. The PPSRI has helped me/my team network with other researchers involved in influenza and pandemic preparedness research.
12. The PPSRI has helped me/my team collaborate with other researchers involved in influenza and pandemic preparedness research.
13. Because of the PPSRI, my research work involves researchers from areas or disciplines that were not involved with my work before.
14. The PPSRI has contributed to a reorientation of my research focus.
15. The PPSRI has helped me/my team involve international research collaborators.
15a. (If "agree" or "strongly agree" to question 15) What are the main roles of your international collaborators? Please check all that apply; Collaborators working jointly on the same problems Collaborators working on different aspects of the same problem Collaborators working on problems pertinent to their setting Other: please specify: _______________________________
15b. (If "agree" or "strongly agree" to question 15) Where are your international collaborators located? Please check all that apply; U.S. Latin America Europe Africa China Other Asia Other: please specify: ________________________________
16. Research users are currently involved in my PPSRI project.
16a. (If "agree" or "strongly agree" to question 16) What types of research users are involved with your project? Please check all that apply. Pharmaceutical company Other type of private sector firm Government Not-for-profit, NGO organization Other: please specify: ________________________________
16b. (If "agree" or "strongly agree" to question 16) How are the research users involved in your project? Please check all that apply; Development of research questions Carrying out some parts of the research Interpretation of the results Application or use of the results Other: please specify: ________________________________
16c. (If "strongly disagree", "disagree" or "neutral" to Q16. Skip if "agree' or 'strongly agree" to question 16 ) Research users will eventually become involved in my PPSRI project.
17. My PPSRI project has a knowledge translation plan in place.
18. My PPSRI project has included interdisciplinary training for students/fellows.
19. My PPSRI project has included mentoring of students/fellows in influenza and pandemic preparedness research.
20. My PPSRI project has increased the number of trainees in influenza and pandemic preparedness research.

Only if role is Nominated Principal Investigator
21. As of December 31, 2008, how many trainees at each of the following levels have been involved in your PPSRI project? (If none, please enter "0")

____ Undergraduate students
____ Master's students
____ PhD students
____ Post-doctoral fellows
____ Other types of trainees (specify)

22. (if the total of Q21 is greater than 0):
Have any of these trainees been involved in international research collaborations?

Yes
No
Don't know

All
E. Continued relevance of the strategies and priorities

In designing strategic research initiatives such as the PPSRI, funding agencies may use different strategies to maximize the chances of significant advancement in knowledge. With respect to the PPSRI, to what extent to do you agree with the following strategies?

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don't Know
23. Funding should ensure that recognized leaders in the targeted research areas can continue to pursue relevant research.
24. Funding should ensure that all researchers who can possibly make a contribution have the opportunity to compete for funds.
25. Funding should concentrate on increasing research capacity by targeting researchers and their trainees who are new to the field.

26. In your view, were there important research priorities NOT included in the PPSRI?

Yes No Don't know

If yes: Please specify which areas were not included:

27. In your view, what are the most important research priority areas for pandemic preparedness research over the next five years? Please rate the existing priorities and add any others you wish to rate.

	Very low priority	Low	Moderate	High	Very high priority	Don't know
Vaccines and immunization
Virus biology and diagnostics
Prevention and treatment
Ethics, legal and social aspects
Others:
1.
2.
3.
4.
5.

F. Background

The following information will help us analyze survey responses according to different types of research and settings.

28. In which of the following disciplines is your main research experience? (Select all that apply)
	Biomedical sciences
	Clinical health sciences
	Health systems and services
	Population and public health
29. With which type of institution or organization are you associated? (Select all that apply)
	University
	College
	University-affiliated hospital
	Community-based hospital
	Community-based/volunteer agency or organization
	Government-funded research agency
	Private research organization
	Other (specify)

G. Suggestions for improvement

30. Do you have any suggestions for improving the PPSRI?

31. Do you have any suggestions for improving other future Institute of Infection and Immunity strategic research initiatives?

Thank you for your collaboration!

[ next section ]