Impacts of the CIHR Institute of Cancer Research 2001-2009

[ Table of Contents ]

Annex A: Strategic initiatives – Detailed review

1. Palliative and End of Life Care

The Palliative and End-of-Life Care initiative has been reviewed in detail separately,¹ and was considered extremely successful by respondents interviewed during that analysis, a conclusion supported by database and bibliometric analysis. This current assessment of the Institute as a whole has confirmed, with a broader group of stakeholders, that the Palliative and End-of-Life Care initiative was not merely a success within itself, but is seen as the major achievement of the Institute since its inception. The Palliative and End-of-Life Care initiative is considered, in fact, to be a model for CIHR and other funders.

Some key success factors for this initiative include:

• The focus on a major health need
• The identification of a significantly underfunded and underdeveloped research area
• The Institute's effective and attractive approach to partnering
• The holistic and multi-faceted approach taken (many components beyond grants and awards)
• The major participation of decision makers as partners, investigators and trainees

A number of respondents, however, expressed considerable concern regarding the traditional funding model for strategic initiatives, whose limitations were highlighted by this initiative: "When we've done something useful, how do we build on that success rather than flitting on to something new? We need to give this more thought." This initiative highlighted many key challenges to the traditional approach that will arise in the support of highly-applied research, such as the recently announced commitment to primary care research.² Future considerations for CIHR include:

• How to evaluate research with local health improvement rather than academic publication as its primary output
• CIHR's responsibility to sustain newly-developed capacity beyond the duration of a single funding initiative
• The need for appropriate mechanisms to support service-engaged health professionals doing part-time research
• The impact of the abolition of open team grants on the effectiveness of strategic investments in emerging team grants: what do they emerge into?

2. Molecular Profiling of Tumours

The Institute's working group on this priority seems to have originally considered the creation of a full-function national tumour bank network, estimated to cost $7.3M³ annually, in the Institute-supported proposal developed by CAPCA. However, at its final meeting, the working group acknowledged that funds on this scale were not available, and instead focused on the two most pressing priorities: "development of a standard operating procedure, and linkage of existing databases."⁴ Working with the Fonds de la recherche en santé du Québec (FRSQ), the Canadian Alliance of Provincial Cancer Agencies, the Ontario Cancer Research Network, the BC Cancer Agency, the Alberta Cancer Board and other parties, the Institute invited an application for the National Tumour Banking Network Grant,⁵ to provide $675,000 per year for 5 years beginning in spring 2004. The result was the Canadian Tumour Research Network, or CTRNet, which has now been funded for five years and is currently under review for renewal. The original aims of CTRNet⁶ were:

• Development of Standard Operating Procedures (SOPs) to provide consistent national standards;
• Creation of a single electronic portal for access to tissues and clinical information;
• Promotion of the exchange of administrative and scientific best practices;
• Promotion of translational research in Canada; and
• Development of a business plan to encourage sustainability.

Stakeholders aware of this initiative all emphasize the growing and central importance of tumour banking in cancer research, and especially in support of translational research. CTRNet, however, appears to be a controversial addition to the cancer research community, and respondents' views as to its utility and effectiveness varied widely from "great success" to "unmitigated disaster." Many concerns seem to relate not to whether CTRNet is doing what it set out to do, but rather whether it ought to be doing something else entirely. This is perhaps not surprising, given the range of activities an "ideal" national tumour repository, or linked network of regional repositories should undertake, and the modest role and funding of CTRNet compared to related international efforts.⁷ CTRNet is currently under review to determine the extent to which it is meeting its stated objectives; we must not second-guess that evaluation. Rather, we will summarize the input received from the key informants regarding the types of objectives CTRNet has met and could potentially meet in the future, and how these relate to respondents' views of Canada's needs regarding tumour banking.

Respondents views on the benefits and achievements of CTRNet

"It's making Canada known as one of the best places for biobanks. Users see the quality difference."

• The Institute was instrumental in spurring the needed connections and collaborations among Canada's tumour banks
• The Institute has made a long term commitment to sustainable funding for this activity
• CTRNet has created standardized protocols, policies, quality control measures and an accreditation process, all now being used across Canada
• Banks are now exchanging best practices and their quality is continuously improving
• CTRNet is undertaking audits of individual banks, leading to improvements or closure, as needed
• CTRNet is now reaching out to "boutiques" and larger banks that are not part of provincial networks
• CTRNet has played a major role in developing international consensus, through the Marble Arch Working Group on International Biobanking, which started by CTRNet, and on which Canada is well-represented
• CTRNet standards have been promulgated internationally,⁸ and are being considered for adoption internationally

Respondents views on the gaps and challenges

"It's not really been a go-to place to get information. Even the website is poor."

• There is still no national bank
• There is still no increased support for the extensive direct costs of banking. The question remains: is this a healthcare cost or a research cost?
• It is not clear whether CTRNet has the clout to improve an individual bank's performance
• CTRNet's actual international impact is unclear
• CTRNet doesn't link banks or provide investigators with a single-window access to samples
• Thousands of individual investigators have small biobanks across the country, which lie outside CTRNet
• The cohort study (The Canadian Partnership for Tomorrow Project⁹) is going to have millions of samples – but there's no national biorepository for them
• CTRNet has no obvious means of raising revenue or ever becoming self-sustaining
• Funded by a grant, CTRNet is being led by researchers and software developers with other full-time jobs, rather than professional management.

Several respondents seem to feel that there could be better ways to use these resources, or to leverage the resources provided to CTRNet. Overall, there is a sense among them that the Standard Operating Procedures (SOPs) developed by CTRNet, while useful, are nonetheless a fairly limited return on a rather large investment, and the single window is still missing. CTRNet's general communications seem to be minimal; much of the concern may simply reflect a lack of awareness of what CTRNet is actually doing.

Some respondents suggested that a much bolder approach is needed; for example leveraging CIHR's funds to do much more: "CTRNet is about SOPs and linkages, not visionary leadership. They could have gone to CFI for a national bio-repository. Maybe the PIs were not thinking big enough about what they could do with that grant, how to leverage it". As noted in "New Funding Needs", CTRNet is an example of the kind of strategic investment which requires the Institute to take a much more hands-on leadership role, rather than remain the traditional hands-off funder: "You can't just fund this sort of thing and walk away for five years and then say 'Oh my God, that's not what I thought we were getting!' It needs to be monitored, there need to be milestones and deliverables. The Institute needs to learn from this how to manage strategic initiatives. They're still treating a strategic investment like an operating grant, and it's not. That's a waste of money."

We conclude that regardless of whether or not CTRNet achieves its defined goals, many stakeholders remain concerned that CTRNet is only a partial fulfillment of the need for a national biorepository network. Furthermore, we note that the Institute has not advanced its involvement in this research priority beyond the establishment of CTRNet, which was intended at the time to be just a first step.

3. Clinical Trials

The Institute made two investments under its clinical trials priority:

1. A "priority announcement" under the open clinical trials competition, to encourage the submission of proposals for studies that focussed on alternatives to treatment that did not involve cytotoxic drugs. The rationale was to encourage clinical trials in this neglected research area.¹⁰ The four clinical trials funded focused on traditional Chinese medicine and aspects of imaging and radiation therapy.
2. A one-time investment of $3.5M in the Clinical Trials Group (CTG) of the National Cancer Institute of Canada (NCIC).

Respondents generally viewed these investments as positive, but of limited impact.

In addition, the Institute held a Drug Development Workshop in December 2002,¹¹ which ended with several recommendations for CIHR action, including the creation of a national network of cancer drug development centres. Few of the recommendations have been implemented.

Non-Cytotoxic Cancer Clinical Trial Priority Announcement

Few respondents were aware of this initiative, and reviews were mixed. Some felt that some important trials were funded that wouldn't have been otherwise, others that these trials rightfully belonged in the open competition. "We got this in place using some the Institute funds and some CIHR funds, and therefore we were able to fund more trials than otherwise, some quite important. But the initiative was clumsily titled, and there was not huge awareness."

Four trials were funded, for a total commitment of approximately $700,000. This investment seems to have had very modest impact. At the time of writing, the results of only one of these trials seems to have been published.

CTG support

"It is a tremendously good investment. The amount of high quality research is astounding; it's one of Canada's best known contributions world-wide."

There is strong agreement among respondents who addressed this initiative as to the value and importance of the infrastructure supported through the CTG, of the synergy which comes from bringing together a national organization, and the need for a consolidated voice at the global research table from an organization with international research credibility. Respondents thought it would be of great value for the Institute to make ongoing investments in building clinical trials infrastructure in general, and preferably in CTG in particular: Canada has proven it has significant and growing strength in clinical trials (see below). There was some question, however, whether NCIC (now the Canadian Cancer Society) would welcome an ongoing partner, as this could dilute its "branding" of this important resource.

The Institute's investment in CTG, while of value, was one-time and therefore of limited overall impact. Respondents do believe, however, that the investment helped the CTG successfully make an important transition.

Despite its prescribed impact, it's worth examining this investment in some detail, because it exemplifies some of the strengths and challenges an Institute faces. A major problem for CIHR, like all federal agencies funded through annual appropriations, is the need to spend all funds by the end of the fiscal year or lose them, regardless of whether an initiative is ready for investment or that the funds will be needed in the next year. This is less of a problem now with a stable or even declining CIHR budget, but it was acute in the early start-up years of the Institutes. Because the Institute had invested considerable time in developing a consensus-based strategic plan across its very large community, it was not ready to flow funds in its first year, and stood to lose most of its budget for that year. The Institute's strength, however, was in its already well-established partnerships, and the flexibility it had at that time to invest its funds by transferring them to another organization.¹² NCIC needed help to get CTG through a major transition, and the Institute was in a position to provide the necessary support. In a decision all agree was opportunistic rather than strategic – but nonetheless a good decision – the otherwise-lapsing Institute funds were transferred to the CTG as a one-time-only contribution.

In selecting clinical trials as a priority, the Institute focused on an area of existing Canadian strength. Canada's number and world share of cancer clinical trials publication have both further improved recently, with a jump in publications occurring between 2004 and 2005 (Fig. A1). Canadian publications from 1999-2000 ranked fifth in the world in terms of citations per publication, and are now first-ranked. Fig. A2 shows the average number of citations per publication in this field of research for Canada and the other leading nations for papers published in 1999-2000 (x-axis) and 2008 (y-axis). The diagonal line shows the average relationship between citations to 1999-2000 publications and to 2008 publications. Those nations with points above the line have improved the citation record for their recent publications relative to the other leading countries, and to the degree that they deviate from the line. Of the five most-cited trials from 2008, numbers 1, 2, and 5 included authors from the NCIC CTG in Kingston. It must be stressed that many of these trials are large multinational, multicentre trials and not necessarily led from Canada. Nevertheless, the fact that Canadians are included in the teams that conduct the trials that attract the greatest number of citations suggests that the Institute was correct to invest in an area of strong and growing Canadian strength. There are, however, concerns that clinical trials for cancer therapy are threatened in Canada.¹³

Fig. A1. Number and world share of Canadian cancer clinical trials publications

Fig. A2. Change in citation performance for cancer clinical trials

4. Early Detection of Cancer

The working group convened to discuss this priority area recommended that the first priority be to support multi-disciplinary research teams to evaluate and compare new cancer screening technologies,¹⁴ though it was recognised that available funding did not meet the need for investment. The Institute invested over $6M in the "Early Detection of Cancer" initiative, mainly through RFAs that funded pilot and team grants for colorectal cancer screening research. In addition, one of the projects funded under the Institute's Access to Quality Cancer Care initiative included a colorectal cancer screening component.

Canadian publications in "colorectal cancer screening" grew a remarkable five-fold since 1999-2000 (Fig A3), obviously not attributable entirely to the Institute's support, since the Institute's funding for screening research only began in FY 2004-05. However, there has been a jump in publication numbers since 2006, which may reflect the stimulus applied by the Institute. The very low world share of Canadian publications in the period 1999-2002 indicates that this was a relatively under-researched area in Canada, and thus a reasonable choice for strategic funding. Perhaps because this area was so undeveloped, there has not yet been an improvement in citation performance to accompany the increased number of publications (data not shown).

Fig. A3. Colorectal cancer screening: Canadian publications and world share

Provisional impacts from the grants funded through this priority are described in an Institute publication,¹⁵ and include two patents and a potential commercialization opportunity. All four of the recipients of the Pilot project grants received follow-on funding from CIHR or another agency. Of the two Team grants funded from 2007-08, the CIHR Team in Population-based Colorectal Cancer Screening has been spectacularly collaborative and productive (Fig. A4). The CIHR Team in Genomic, Imaging and Modeling Approaches to Advance Population-Based Colorectal Cancer Screening has not yet published any team-based papers, though six publications by the nominated PI are credited to this grant.¹⁶ We consider that the output from this strategic investment has been as expected for a modest investment.

Fig. A4. Co-publication record of the members of the CIHR Team in Population-based Colorectal Cancer Screening before and after receiving the grant. Each circle represents one member of the team, and a line linking circles indicates that those team members are co authors on a publication. The thick line linking #1 and #9 represents 14 publications.

Although the bibliometric data suggest growing output in research on colorectal screening, respondents aware of this initiative generally thought it was a well-intentioned idea but ultimately had limited impact overall in addressing the Early Detection priority, with three separate RFAs¹⁷ resulted in a total of only seven applications for pilot projects (four funded), and five for teams (two funded). Respondents cited a number of reasons for the initiative's relatively low impact:

• The funds available were too small to make a notable difference in what research was being done.
• The priority was too broad, and needed to be targeted to an area where research was specifically needed.
• The timing was wrong: there was a lack of specific, researchable questions with potential and real impacts
• The process was low-key, lacking both community awareness and perhaps the community input which would have resulted in a more targeted and attractive initiative.

"The sense was, at the time, we could not put on the table some strong, convincing, exciting initiative. That's not true today."

Overall, most respondents seemed to believe that the Institute was well-advised to consider this initiative low priority for the future, and focus its resources elsewhere. However, some respondents noted that the environment has changed, and there are clear and important niches to be addressed now in this overall area of early detection, such as around using modelling, new technologies and new screening areas such as lung cancer, and how to implement newly-available evidence around colorectal cancer screening.

5. Risk Behaviour and Prevention

The working group tasked with developing actions under this priority "recognized from the outset that the appropriate way to proceed in developing a research agenda on risk behaviour and prevention was to build on existing, partnerships, infrastructure and alliances."¹⁸ The Institute therefore addressed this priority in 2003 by investing $480,000 per year for 5 years in CIHR's Tobacco Abuse and Nicotine Addiction initiative, coordinated by the Canadian Tobacco Control Research Initiative (CTCRI). CTCRI represented multiple partners, including several CIHR Institutes, the Canadian Cancer Society (CCS), NCIC and Health Canada. With a number of CIHR Institutes and other partners, a request for applications in "Advancing the Science to Reduce Tobacco Abuse and Nicotine Addiction"¹⁹ funded three team and 29 other grants over the period July 2004 to the present. CIHR's investment in CTCRI to date has been $7.7M.

"CTCRI – how could we not fund it?"

Because the Institute invested through an external organization, CTCRI, this initiative has remained low-key, and is mostly unknown outside of the tobacco control research community. However, those respondents aware of the CTCRI felt that the Institute's contribution was essential to keep the initiative going, and partners in particular were extremely appreciative of the Institute's willingness and ability to invest collaboratively with them in an external initiative. The Institute was also an important champion of tobacco research, and helped engage a number of other institutes and partners in CTCRI.

Stakeholders note that this type of policy research is ill-suited to open grants competition support, and thus the Institute's support of a different kind of external initiative, though it gains it little credit or profile for the Institute, makes an important contribution to improving health.

Fig. A5. Tobacco control: number and world share of Canadian publications

While some respondents were concerned that there is simply no information available upon which the outcomes of the CTCRI could be judged, others believed that CTCRI did, ultimately, create substantial and self-sustaining research capacity, as well as a few key world leaders: "The country has had very little capacity – and we are now a world leader in this research.… It's not just the Cancer Society – the Institute was huge in getting this to happen."

The assertion that Canada is now a world leader in the field of tobacco control research is supported by publication data. Fig. A5 shows that Canadian publications in the area of tobacco control, though few in number, have increased significantly since 2005, as has Canada's share of world publications.

Fig A6 compares citation performance for Canadian tobacco control papers published in 1999-2000 and in 2008. Canada's recent publications have improved considerably, and Canada now ranks second in terms of citation frequency, compared to fourth for publications dating from 1999-2000. It would seem probable that this is the result of significant investment in tobacco control research by the Institute and its partners.

Fig. A6. Tobacco control: changes in citation performance between 1999-00 and 2008 publications for Canada and other leading nations. The diagonal line shows the average relationship between citations to 1999-2000 publications and to 2008 publications. Those nations with points above the line have improved the citation record for their recent publications relative to the other leading countries, according to the distance that they deviate from the line.

Since this seems to be an area in which Canadian research is improving, and is relatively circumscribed in terms of the number of publications, we explored it in more detail. We found 12 Canadian publications among the 100 most-cited publications from 2006-08 (the top-ranked #11), compared to only three of the 100 most-cited publications from a decade earlier (the top-ranked #50). Two Canadian authors rank #3 and #4 on the list of most-cited authors of 2006-08 publications, and one, Geoffrey T. Fong²⁰ of the University of Waterloo, and Chief Principal Investigator of the International Tobacco Control Policy Evaluation Project, ranks #1 (equal with two Americans) in terms of h-index for 2006-08 publications, a composite measure of scientific impact that takes into account both the number of papers published during that period and the times they have been cited.²¹

We conclude that the Institute's investment in CTCRI has had a significant impact in terms of sustaining the overall partnership and leveraging additional funding, with the result that Canadian research in this area has achieved world-leading status.

In 2009 the CTCRI partners concluded that "CTCRI has fundamentally succeeded in what it was charged to do – build the tobacco control research community and support high quality, high impact research,"²² and declined further funding. It will be interesting to see if the momentum in research productivity is sustained without this targeted funding.

6. Molecular and Functional Imaging

The Institute's working group "recognises that although there have been significant advances in imaging technology in recent years, there is still an urgent need for improved imaging and spectroscopy devices, contrast agents, radiopharmaceuticals and optically-labelled probes."²³ As a result of a 2003 RFA, the Institute supported the Institute supported four teams for short terms of two years, for a total investment of over $1.3 million.²⁴ The Institute's own interim assessment was that three of the teams had achieved significant technical advances as a result of these grants, including three resulting patent applications and two start-up companies. One of these companies was subsequently acquired by a larger US company,²⁵ but it retains a presence in Canada. One of the teams was able to leverage the Institute grant into much greater funding from CIHR and other sources. In terms of publications, however, the output has been small, with only one or possibly two publications identified as resulting from this funding so far.

Only a few respondents felt qualified to comment on this initiative, and their views were mixed:

• The scope of the initiative was too small; it needed a broader consortium of stakeholders.
• There was no community ready at that time, or the Institute targeted the wrong community
• Imaging lacked a strong champion at the IAB, so made little headway.

The June 2009 RFA on Alternative Radiopharmaceuticals for Medical Imaging, launched in response to the lack of medical isotopes after the shutdown of the National Research Universal (NRU) nuclear reactor at Chalk River, went virtually unnoticed by most members of the cancer research community, perhaps deliberately, given the one respondent's view that "I'm actually embarrassed by it," seeing it as a political decision by CIHR, made without the prior knowledge of the Institute Advisory Board.

However, the currently developing discussions on future opportunities for imaging research, launched through an October 2009 workshop²⁶ organized by the Institute, seem to hold some promise to better engage with the community around this priority and find a more effective role for the Institute to play in the future.

7. Access to Quality Cancer Care

The Institute sponsored a workshop on this issue in June 2005. A key recommendation was to "create new interdisciplinary teams designed to build linkages between researchers, health care providers, patients and decision makers and unsure the timely uptake and implementation of research findings."²⁷ The Institute has committed about $8 million to support research into Access to Quality Cancer Care, with considerable additional funds provided by partners.²⁸ The initiative has included two "rapid response" RFAs on wait times, in 2004 and 2005, and more recently, the funding of seven New Emerging Team (NETs) grants, launched in 2007 and now at the mid-point of their five-year grants, and one Interdisciplinary Capacity Enhancement (ICE) Grant, funded in 2006. The authors of this report had the opportunity to join an October 2009 workshop where the NETs discussed their successes, challenges, and needs to achieve their goals and future sustainability, as well as talk with many of the participating NET PIs, investigators, trainees, staff and decision-maker partners in addition to those formally interviewed.

Canada's share of world publications (recognizing that "Access to Care" is a difficult topic on which to conduct a bibliometric search) was a high 8-9% throughout the period studied and showed no upward or downward trend. Its citation rate, however, seems to be declining, from a high of first place in 1999-2000 to fourth in 2008 (Fig. A7). Because of this unexpected result, we repeated the analysis, searching the cancer literature instead on the Web of Science classification terms that best describe health services research.²⁹ There was steady growth in publications, again with most occurring since 2002, and world share had also increased steadily from about 4% to almost 6%, as expected for an area in which Canada is highly specialized. However, citation performance also showed some deterioration relative to other leading nations, and Canadian publications, both from 1999-2000 and from 2008, ranked eighth out of ten in terms of citation frequency. Thus while Canada has significantly increased its production of health services-related publications in the cancer field, this has yet to result in improved citation performance, and relative to other leading nations, Canadian research seems to have lost ground in terms of citations. It will be interesting to see if the publications emerging from the Access teams, as they move into a mature phase, will reverse this trend.

Fig. A7. Cancer care: changes in citation performance between 1999-00 and 2008 publications for Canada and other leading nations. The diagonal line shows the average relationship between citations to 1999-2000 publications and to 2008 publications. Those nations with points above the line have improved the citation record for their recent publications relative to the other leading countries, according to the distance that they deviate from the line.

Because the teams are only at mid-point of their funding we did not conduct an analysis of their publications. It is also unlikely that the Access NETs would have yet had any impact on Canada's publication record.

However, highly-cited academic publications are less important in this area of research than the actual impact that the results have on the organization and quality of cancer care. In this respect, many respondents pointed to Access and Palliative Care together as models of how to run a good initiative, noting important common components such as: identifying an area of high need where additional funds can make a real difference, working with well-coordinated partners, and taking a broad approach with a preliminary planning workshop to identify directions and needs. In one respondent's view, both these initiatives are attempting to make the shift from "funding projects to funding environmental change."

Many respondents emphasized the need for the Institute, and CIHR in general, to focus more on identifying what it takes to create research which is not merely scientifically meritorious, but which can and will be implemented by knowledge users. Another major challenge is to work with the health care system to create research "pull" , i.e. a demand for CIHR-funded research, demonstrating to decision makers that investing in and using research helps solves the problems that keep them awake at night.

Working in a New Emerging Team (NET)

A key aspect of the Access teams, like NETs we have interviewed elsewhere, is the novel insight that comes from the cross-pollination of ideas: "We supplement each others' ideas, generate new questions. We find insights we couldn't have had without these interactions." Research becomes more meaningful as the team shares their understanding of "what's clinically important vs. what's measurable."

Team grants provide a unique platform, a breathing space even, to focus on activities that are usually peripheral to a busy researcher: "It's a huge advantage in team grants – it provides an environment and platform for building relationships. It's a big group; I doubt we could hold it together without the infrastructure." Many stakeholders generally had noted that applied research and KT are more about environment change than research in the traditional understanding of it, and one Access team member noted: "Changing cultures, ideas - these are not things you do in operating grants!"

Like NETs elsewhere, the Access teams have found NETs fertile grounds for training. "Trainees get broad perspectives, new methodologies and contexts, new ways to do things, interactions and contacts. To me, it's a unique opportunity to build capacity in areas of significant challenge. Ours has not been a research-intensive discipline; the team environment is a huge benefit." Trainees find that with multiple supervisors and perspectives, they can ask questions that would otherwise be impossible to address. They have access to people, material, data and experiences unavailable to their colleagues in traditional training environments. Trainees also really appreciate the chance to see the kinds of use and impact their work could have: "It's very exciting for a researcher - you can see the potential for huge application of what you do."

Challenges for the NETs

There is significant hope among respondents that the Access teams will prove to be as pivotal to that research community as the palliative care teams have been in theirs. However, while it is far too early to judge the ultimate success of the Access teams, respondents have reservations about the likelihood that this initiative will achieve its full potential. There seem to be two major concerns:

1. Short-term focus: The NETs seem to have used their funding more to support short-term activities, such as pilot projects, than to make long-term investments in building relationships, research capacity and infrastructure. This approach is an understandable response to being clearly told there is no potential for renewal of the NET grant, nor is there any longer an open team grants competition to which they might apply when the strategic funding runs out. Researchers adjust their priorities accordingly: "I watched [another NET] end. It was probably ten times more successful as an integrated team; it's a huge problem finding follow-on funding." Unfortunately, it means these teams (in contrast to those we encountered in the Palliative and End-of-Life Initiative, who assumed there would be follow-on funding opportunities), are investing less time as well as money in the more transformative components of the teams' activities: building the relationships among team members, decision-makers and other stakeholders; creating knowledge translation capacity; and training and mentoring new investigators.
2. Knowledge Translation (KT) gap: The Access teams are building multidisciplinary links across research communities, but most teams have not yet developed very strong links with users, or integrated them into strategic planning and activities. "It's a platform for building relationships across researchers at least; the KT relationships are still challenging." While the RFA required decision-maker involvement,³⁰ applicants were provided with little guidance in how to achieve it; they note "in KT, it's about relationships. This is the hard part – most people are successful in the research part." At their October 2009 meeting,³¹ the Access teams expressed considerable uncertainty as to what was expected of them in the realm of KT, and how to achieve it. There was little awareness among many teams of the types of tools or resources – such as knowledge brokers – which might help them achieve their goals.

"It would have been nice to have this meeting two years ago. The horse has left the barn."

A five-year team is a major investment. The Institute – and CIHR more broadly – has an opportunity to increase its return on investment in teams by providing them with expanded guidance and structure. It would be timely to examine CIHR's experience to date with teams, and identify the types of goals that teams can achieve, and the types of resources they need in order to be able to so. If CIHR is going to continue to provide grants with the intent to create multi-disciplinary research teams operating in an integrated knowledge translation mode, it needs to provide support, including concrete expectations, how-to's, training and resources to those teams to help them achieve those goals. It would not be unreasonable to suggest, for example, that a knowledge broker be a core component to every new NET – but if so, CIHR needs to ensure that emerging teams know what a knowledge broker is, how to use one, and where to find one.

8. Research Training

In 2001 and 2002, CIHR and partners launched the Strategic Training Initiative in Health Research, to fund transdisciplinary, integrative health sciences research training programs known as STIHRs. As a result of the first two launches, 86 training programs were funded CIHR-wide, for six years each at a maximum of $325,000 per year. As training had been identified as a priority for the Institute of Cancer Research, the Institute engaged a wide range of partners to collectively increase the funding for cancer STIHRs by $13.3 million; as a result, 22 of the funded STIHRs were in cancer-related areas, although only eight of them identified ICR as their primary institute of affiliation. The uptake of this CIHR program was so unexpectedly high that without the additional funds provided by the Institute and partners, there would have been only 6, not 22, cancer STIHRs.

The second round of the STIHR program was launched in January 2008, and the Institute committed $6 million over the next six years to support STIHR programs with a focus on cancer research. Five training programs affiliated with the Institute were funded, of which three had cancer as their major focus (the other two were on synchrotron technology and drug research, with cancer as an important field of application). Four more STIHRs were funded in cancer-related areas, though with other Institutes as their primary affiliation. In addition, through a partnership with the Terry Fox Foundation, a further four programs were funded, all primarily affiliated with the Institute. Of these 13 STIHRs, two are new, and 11 are renewals from among the original cancer-related STIHRs, giving the Institute a slightly better renewal rate than the overall competition success rate of 45%. Thanks to the partnership with the Terry Fox Foundation, there are now nine STIHRS affiliated with the Institute, compared to eight in the first round. Other Institutes did not fare as well: for example, the ICRH went from 11 to four affiliated STIHRS, and INMHA from 11 to six.

Benefits and challenges of STIHRs

We have consistently received significant positive feedback from a wide range of STIHR trainees, and the cancer STIHRs are no exception. They have many of the same major advantages of training within a NET team, with perhaps a bit less involvement in the practical details of others' day-to-day research activities, but with more structured formal training and courses. Each STIHR seems to be different, providing unique advantages and opportunities to trainees. For instance, trainees from one cancer-based STIHR were particularly impressed with a course on public and private partnerships in research and oncology, which covered subjects ranging from working with industry to developing a drug through regulatory approvals. Another trainee chose a cancer STIHR which allowed each trainee to undertake three short projects in three rotations – anywhere in the world – before choosing a supervisor and topic.

STIHRs have a big role in spurring translational research. Students found the cancer STIHRs permitted them significant contact with clinicians, helping them prepare for – or even consider choosing – a clinician-scientist career path rather than a purely academic one. A mentor describes how in a STIHR environment "molecular biologists see how their work makes an impact on patients and how each researcher fits in to the bigger picture. It's very useful." Other trainees were involved with STIHR administration, and developed early some of the key skills required of a researcher, such as how to review a research proposal.

Overall, trainees find their experience broader than that of their peers in conventional disciplinary programs, making them better able to choose their next steps, and giving them the skills, knowledge, experience and contacts they need to attain the next steps they would choose. "There are more interactions with mentors, and more mentors."

Other stakeholders also speak very positively about the STIHRS, seeing them as a good investment and an effective approach to capacity building. PIs noted the value in being able to attract students quickly, rather than having to wait on annual award funding. They also noted that STIHRs, unlike individual awards, "create a community of learners", and have significant translational impact on the PIs involved, and not just the students. "The enthusiasm demonstrated by our stakeholders shows the value and success of this approach. The STIHR is the glue that holds people together and gets researchers talking to each other. Charities, hospitals, faculties of medicine, Ministries of Health – all endorsed the application. Business too."

A few stakeholders expressed some reservations about STIHRs, usually in response to exposure to a particular example of a less-than-optimal STIHR: "CIHR is getting better at identifying the good ones and getting the funding to students rather than to professors." CIHR and the STIHRs more broadly would benefit from the collation of best practices and experiences of the many STIHRs funded to date, and clarifying some of the expectations and ideals for effective STIHRs. It would also be timely for CIHR to conduct a thorough analysis of the performance of graduates from STIHR programs compared to a grade- and discipline-matched sample of graduates from conventional training experiences. Have the STIHRs actually provided an experience that helps students better to succeed in the current research environment? Do graduates practice a different, more collaborative and interdisciplinary type of research? Do supervisors gain benefits from participation in STIHRs that offset a greater time commitment? Do conventional programs have advantages for certain types of program, student, or career pathway?

Breast and prostate cancer

While strictly speaking not a priority of the Institute, these two areas of cancer research have received targeted funding from CIHR through specific partnered initiatives, and CIHR's representation on the governance of these initiatives has been through the Institute. In the case of breast cancer, the Canadian Breast Cancer Research Alliance (CBCRA) and its predecessor Initiative have been in existence since 1993. The total investment by all funding partners up until the end of FY 2007-8 was $178M,³² an average of $12M a year. CIHR's contribution has varied, but has been as much as $5.5M/year (in 2005-6).

The Canadian Prostate Cancer Research Initiative (CPCRI) ran for ten years, terminating in 2009. The website no longer exists, but a dubious partial remnant³³ provides some information. From 1998 to 2004 over $5M was committed by the partners to this initiative ($1M/year), as compared to the approximately $12M annually to the breast cancer initiative.

Fig. A8. Breast cancer: number and world share of Canadian publications

Fig. A9. Breast Cancer: changes in citation performance between 1999-00 and 2008 publications for Canada and other leading nations. The diagonal line shows the average relationship between citations to 1999-2000 publications and to 2008 publications. Those nations with points above the line have improved the citation record for their recent publications relative to the other leading countries, according to the distance that they deviate from the line.

Publications in breast cancer show a steady rise from 2000 onwards, with an apparent recent levelling off (Fig A8). World share has remained relatively constant at a high 5-6%. Publications from 1999-2000 with Canadian authors are the most-cited out of all the leading nations in this field, and this is true of 2008 publications also (Fig. A9). It would appear that long-term support of breast cancer research in Canada through the CBCRA and its predecessors has resulted in the production of world-leading publications, stability in research output, and maintenance of Canada's strong competitive position in this field.

Prostate cancer publications have increased markedly (Fig. A10), with the increase following by 2-3 years the formation of the CPCRI (though this was so modest that it is unlikely to be the sole reason for it). Similarly to breast cancer research, there is evidence of a recent stability in numbers of publications. World share has increased slightly. Again like in breast cancer research, Canadian publications are highly cited relative to those of other leading nations, with those from both 1999-00 and 2008 being the second most highly cited (Fig. A11).

Fig. A10. Prostate cancer: number and world share of Canadian publications

Fig. A11. Prostate Cancer: changes in citation performance between 1999-00 and 2008 publications for Canada and other leading nations. The diagonal line shows the average relationship between citations to 1999-2000 publications and to 2008 publications. Those nations with points above the line have improved the citation record for their recent publications relative to the other leading countries, according to the distance that they deviate from the line.

The Canadian Partnership for Tomorrow Project (the CPAC cancer cohort)

Several respondents pointed to CPAC's newly-developed cancer cohort as a major Institute accomplishment, although its formal relationship to the Institute is rather tenuous: "With the cohort, it's hard to know the Institute's role in making it happen. But Phil (Branton) was instrumental, with his CCRA hat, to get CPAC to invest its money there." It is clear that Dr. Branton, as founding Scientific Director of the Institute, was able to play a large role in many external activities because of the respect accorded to his leadership role at the Institute – as well as his inimitable personal qualities. Thus activities like the cohort, though not led by the Institute per se, were none the less possible because of the contributions of the Institute's leadership.

When the Canadian Partnership Against Cancer (CPAC) first established its $50M five-year research budget, it envisioned quite a different approach with its investment – something more like $10M in total to the cohort, and the rest to translational research. The Canadian Cancer Research Alliance (CCRA) was critical in persuading the Partnership to focus largely on fully supporting the cohort, and letting other agencies, especially the Terry Fox Foundation, to take the lead in other types of translational research, as they were already actively investing new funds in that area. "The importance of CIHR was in championing the cohort, promulgating the message of why it's important and re-positioning people to understand the concept of investing in a 15 year project, giving its scrutiny, imprimatur."

Respondents identified a number of outstanding questions associated with the cohort, their largest concern being its sustainability: the current funding is only a five-year commitment. What happens if the government funders begin to withdraw their support – not too difficult to imagine in the current economic climate – will CIHR be expected to pick up the bill? The cohort is expensive, and taking on a portion of its annual cost could significantly reduce the amount of CIHR's budget available for new commitments each year. Respondents are also concerned that "there was not enough forethought given to the cohort," in particular, to "what will make it unique, and therefore justify its funding. It's going to produce a lot of data, but I don't think we have the community available to use the data – these people aren't ready, existing, knowledgeable."

The cohort is an extremely large investment in a non-traditional type of scientific platform whose ultimate scientific output is yet to be determined. Not surprisingly, this investment is controversial, with respondents considering it anything from an extraordinary and unique opportunity to a colossal waste of money. Its champion, Dr. Branton, himself "came into it kicking and screaming, and then became very supporting." Whatever one may conclude about the ultimate benefits of investing in a large cancer cohort, several conclusions are clear: the cohort will shape the direction of considerable research investment over the next years, and quite potentially, given its largely provincial funders, health care as well. For the purposes of this report, the conclusion is that the cohort would probably not exist – certainly not so soon or so large - without the active championship and backing of the Scientific Director of the Institute of Cancer Research.

Concerns about sustainability are common to all large cohort studies in Canada (e.g. the CHILD study³⁴ and the Canadian Longitudinal Study on Aging³⁵). All were started with just a fraction of the funding they need for completion, and all are busy seeking the funding shortfall. This habit of launching major projects without a source to sustain them through the long term, when they will really pay-off scientifically, will become a major problem for CIHR, which will find itself pressured to take on the funding responsibility so these projects do not fail. It is both scientifically and ethically irresponsible to abandon a cohort study that is delivering useful data.

Annex B: CIHR's application pressure in cancer research

Concerns about established investigators

Several respondents expressed concern that the growth of the cancer research community has increased competition for operating grants to the point where established researchers are no longer being funded: they fear that an influx of outstanding new researchers displaces applications from the previously-funded and merely excellent to below the funded range. This would defeat the whole point of increasing capacity, and so we tried to determine the extent to which these fears might be justified.

First, we note that there is an automatic compensation in the CIHR open operating grant competition for the growth of a research community: since applications to all CIHR peer review panels are funded to the same merit percentile, more cancer applications mean more cancer grants funded, at the expense of other areas of research.

The fact that the same percentile of applications is funded by each review committee (with small adjustments in order to fund integers) allows us to derive application pressure from the number of grants awarded. We approached the analysis in two ways, both of which were partly compromised by missing CIHR data, or changes in the way competition results are reported. First, we used the number of awarded grants listed by primary Institute affiliation. Second, we used the number of grants awarded that were reviewed by the three dedicated cancer research review committees (CBT, CPT and MCC). Both approaches have their limitations: there are cancer-related grants that are not affiliated with the Institute, and many cancer-related applications are reviewed by committees other than the three noted above. In fact, for the latest competition, cancer-related applications (as determined by keyword search) were reviewed by 27 different committees, with the three dedicated committees accounting for about one-third.

Fig.B1 shows that there is a trend of increased application pressure for cancer grants when they are designated by affiliation to the Institute: the percentage of total applications so affiliated rises from 12 to 15% over a 3.5 year period. No increase is evident for the applications reviewed by the three cancer committees, but as noted above relevant applications are reviewed by many other committees, and the officers of the cancer committees could divert some of their increased workload to other suitable committees.

Since there is some evidence for increased application pressure, we examined the CIHR grant fate of individual researchers in cancer, and across all areas of health research. Out of a sample³⁶ of 426 operating grant PIs in 1999/00 from all areas, 53.1% were still holders of open operating grants in 2008/09. Out of all operating grant PIs in 1999 who chose "cancer" as one of their keywords (401), 49.9% were still holders of "cancer" operating grants in 2008/09. One of the trivial reasons why cancer grantholders may have disappeared between 1999/00 and 2008/09 is simply that they stopped using the keyword "cancer": maybe they changed research fields. We were able to identify such individuals because they appeared in the first "all area" sample, and we estimated that of the 401 cancer grant holders from 1999, there were 15 who had changed field and no longer used "cancer" as a keyword, but still held an operating grant. When those individuals are added to the "cancer" grant holders, the percentage of the 1999/00 grantholders who still hold operating grants in 2008-09 goes up to 53.6%. We conclude there is no evidence that long-established investigators in the cancer area are losing their grants at a faster rate than those in other fields of health research.

Incidentally, the high percentage of those who held on to their funding from 1999/00 to 2008/9 means that the success rate for these investigators when they apply/reapply for their operating grants has been between 75-80% (depending on the duration of the grant) over this period, compared to the overall success rates for all applicants which varied from about 35% to 16% in the competitions over the same period.

Fig. B1. Cancer applications to the open operating grants competition as a % of all applications

Fig. B2. Changes in institutional funding between 2000-01 and 2009-09

Concerns about larger institutions

In some interviews we heard concern that the increase in CIHR funding for cancer research had been accompanied by "the rich getting richer", i.e. that institutions already performing well in cancer research were the major beneficiaries of the increased funding, at the expense of institutions that were developing their expertise in cancer research. Fig. B2 shows there is no factual basis for this concern. The percentage of total CIHR funding for cancer research (keyword search) flowing to institutions in 2000-01 is plotted against the percentage in 2008-9 for the same institutions. Those institutions that lie below the dotted diagonal line have gained share of CIHR funding, and if anything these tend to be the smaller institutions.

---

Appendix References

1. CIHR Institute of Cancer Research "Impacts of the Palliative and End-of-Life Care Initiative 2003-2009".
2. Webster, P. "CIHR pledges to tackle primary health care" CMAJ News Jan 20, 2010.
3. CAPCA Tumor Bank Working Group "Models for Tumor Bank Programs in Canada".
4. The Institute Molecular Profiling of Tumours Working Group: Second National Tumour Banking Network Planning Meeting, October 2003.
5. National Tumour Banking Network Grant RFA.
6. CIHR-ICR "Molecular Profiling of Tumours".
7. The UK National Cancer Tissue Resource, launched by the National Cancer Research Institute in 2003 was provided with approximately three times the funding of CTRNet. The National Cancer Institute in the USA has recently received US$70M to support its human tumour biobank (caHUB) from the American Recovery and Reinvestment Act.
8. Riegman, P.H.J. et al "Biobanking for better healthcare" Molecular Oncology 2: 213-222 (2008).
9. The Canadian Partnership for Tomorrow Project Factsheet.
10. CIHR-ICR Minutes of the 1st Meeting of the Research Priority Working Group on Clinical Trials, June 21st, 2002.
11. ICR Clinical Trials Working Group – Drug Development Workshop Report.
12. Changes to the Federal Government (Treasury Board) rules that determine how grant funds can be spent have subsequently removed this flexibility.
13. Globe and Mail editorial "Ill-funded public good" February 11, 2010.
14. CIHR-ICR Minutes of the 1st Meeting of the ICR Research Priority Working Group on Early Detection of Tumours, December 6, 2002.
15. ICR Early Detection of Cancer.
16. CIHR Team in Genomic, Imaging and Modeling Approaches to Advance Population-Based Colorectal Cancer Screening.
17. ICR "Early Detection of Cancer" (listing of RFAs).
18. CIHR-ICR Minutes of the 1st Meeting of the ICR Research Priority Working Group on Risk Behaviour and Prevention October 11, 2002.
19. Advancing the Science to Reduce Tobacco Abuse and Nicotine Addiction – Interdisciplinary Capacity Enhancement Grant Program (Archived) Request for Applications.
20. CIHR News Releases "The Science Behind Combating the Greatest Threat to Global Health: Tobacco Use" 2009.
21. Dr Fong's h-index for his 2006-08 publications is 12, meaning that 12 papers from this period have been cited 12 times each.
22. CTCRI – Message from the Board – 11 March 2009.
23. CIHR Request for Applications "Novel Technology Applications in Health Research".
24. ICR "Molecular and Functional Imaging".
25. Gamma-Medica Ideas Inc.
26. CIHR-ICR: Workshop report pending
27. CIHR-ICR Access to Quality Cancer Care – Workshop Report June 14th, 2005.
28. CIHR-ICR "Access to Quality Cancer Care".
29. Health Care Sciences & Services; Health Policy & Services
30. CIHR Request for Applications "Team Grant: Access to Quality Cancer Care" December 2005. The requirements included: "It is particularly important for this RFA, that research teams include health care providers and health system managers operating in a policy/ decision making capacity and have a clear plan for effective and timely knowledge translation"
31. CIHR-ICR "Access to Quality Cancer Care Workshop, Oct 8-9, 2009: Workshop Report".
32. CBCRA Annual Report 2007-8.
33. Use caution: others have reported issues with this website.
34. The Canadian Healthy Infant Longitudinal Development (CHILD) Study.
35. The Canadian Longitudinal Study on Aging.
36. Those with family names AAAA to CON