Impacts of the CIHR Institute of Cancer Research 2001-2009

[ Table of Contents ]

Introduction

About the CIHR Institutes

The Institute of Cancer Research (ICR) is one of the 13 virtual institutes that make up the Canadian Institutes of Health Research (CIHR), established in 2001. Each institute has a Scientific Director, supporting staff, and an Institute Advisory Board which consists of leading researchers, partners and research users.

When investigators apply to CIHR programs, they self-select an Institute to which each grant or award seems most closely affiliated, thereby creating a pool of researchers associated with each Institute. In addition, each Institute has a dedicated budget with which to address the strategic research priorities identified by its Director and Advisory Board. This budget has grown from about $500,000 per year per Institute to its current $8.4M per year.

About the CIHR Institute of Cancer Research

The Institute of Cancer Research is dedicated to supporting research that reduces the burden of cancer on individuals and families through prevention strategies, screening, diagnosis, effective treatment, psycho-social support systems, and palliation.

When the Institute developed its inaugural Strategic Plan in 2002,¹ it included six strategic orientations. Since the Institute used similar approaches to address several of these objectives, we reviewed the Institute under four merged categories, as listed below:

Strategic Orientation	See section:
Play a leadership role in setting the Canadian strategic cancer research agenda.	D. Building a national research community
Lead in co-ordinating funding of cancer research by all national, provincial and local organisations in Canada
Support and develop cancer research initiatives, programmes and projects that respond to that agenda.	E. Delivering ICR's Initiatives and activities
Develop and support capacity building in cancer research
Seek collaboration/partnership with other cancer agencies and stakeholders.	F. Building partnerships
Stimulate research required to facilitate dissemination, transfer and translation of knowledge from research findings into policies, interventions, services and products	G. Translating knowledge into health improvements

In conjunction with its 2002 Plan, the Institute also developed eight research priorities, which are addressed briefly under "Delivering ICR's Initiatives and activities," and in detail in Annex A (to which the hyperlinks in the list below will divert the reader).

1. Palliative and End of Life Care
2. Molecular Profiling of Tumours
3. Clinical Trials
4. Early Detection of Cancer
5. Risk Behaviour and Prevention
6. Molecular and Functional Imaging
7. Access to Quality Cancer Care
8. Research Training

About this impact assessment

The Institute has now completed its inaugural strategic plan, and wishes to assess the successes, challenges, and lessons learned from the experience of creating and launching this new approach to supporting cancer research in Canada. It seeks to evaluate its overall performance in the cancer community as well as its contributions to furthering CIHR's mandate.

The Institute is now in the process of developing its second Strategic Plan, to be launched in 2010. To inform the implementation of its new strategic plan, the Institute seeks to understand how it can most effectively incorporate its best practices, identified strengths and most effective approaches into future planning, and address the challenges which remain.

The authors of this report have previously undertaken an assessment of one of the Institute's major strategic research initiatives, Palliative and End-of-Life Care.² We will not repeat that analysis here, but will focus primarily on the Institute's strategic orientations and other priorities.

The objectives of this review are to assess:

1. The Institute's role in and contributions to the Canadian and international cancer research community.
2. The degree to which the Institute's research priorities have/are achieving their objectives.
3. Changes in the Canadian cancer research environment since 2000, and the degree to which they can be attributed directly or indirectly to the Institute.
4. The Institute's strengths and remaining challenges, which should underpin its new strategic plan.

The assessment process

The assessment blends quantitative and qualitative approaches, using multiple sources of data to compare and contrast the evidence of impact. The major components of this analysis include: an environmental scan of CIHR, partner and stakeholder documents; database analysis including those of CIHR and other key funders, as well as public and proprietary sources of bibliometric data;³ and interviews with about 40 key informants, including cancer researchers, leaders of Institute activities, representatives of other funders of cancer research in Canada, and other stakeholders in cancer research. Specific anonymized comments from the informants are presented in the text in inverted commas, and, where they seem particularly apropos, in text boxes.

Our primary goal is to identify the impacts of the Institute, that is: in what ways is cancer research in Canada different because of the creation of the Institute, and what are the implications of those changes? Our focus is therefore not on all cancer research and researchers, but specifically on those areas which have been affected by the Institute's activities to date. We begin by looking at the environment in which the Institute has had to operate.

Canadian cancer research funding: then and now

2000: Two large funders

Each of CIHR's thirteen institutes was created with unique opportunities and challenges. While some had to contend with small or fractured research communities, the Institute of Cancer Research had the opposite experience: its constituency was a large, well-developed, well-funded, and mature research community, supported by many large provincial and charitable funders.

Foremost among these was the National Cancer Institute of Canada (NCIC),⁴ large enough to propose, during CIHR's early planning stages, that a joint Institute be created under NCIC rather than establishing a new and potentially duplicative cancer institute under CIHR. Investing $42M in research in 1999-2000,⁵ NCIC easily matched the former Medical Research Council's cancer investment of $38M in the same fiscal year.

Other significant players included regional and disease-specific cancer charities, such as the Terry Fox Foundation and the Canadian Breast Cancer Research Foundation. In addition, and unlike other areas of health care, many provinces have integrated cancer care agencies which perform intramural research. All told, when CIHR's new Institute of Cancer Research was launched with an annual strategic budget of about $500,000,⁶ the total funding for cancer research in Canada was probably in the vicinity of $100M-$150M per year.

2009: Shifting the centre of gravity

Peer-reviewed research funding

The Canadian Cancer Research Alliance (CCRA) has undertaken extensive work to identify all the peer-reviewed research funding invested in cancer. In its latest report,⁷ CCRA found that by 2007, investment in peer-reviewed grants had increased to over $400M per year (Fig 1), representing a three- to four-fold increase in seven years. While CIHR is the single largest funder, it only represents about 30% of the total. This is very different from most other areas of health research, where CIHR provides the clear majority of Canadian funds. CIHR's investment (Fig. 2) is now much larger than the $50M invested in 2008 by the Canadian Cancer Society Research Institute (successor to NCIC), though the CIHR's Institute for Cancer Research controls only ~$8M of that budget (a contentious issue for partners which will be discussed further below).

The CCRA survey lists 38 funding organizations. The top eight provide 82% of the total funding (i.e. an average of $40M/year each, equivalent to what MRC and NCIC were each spending in 1999-2000), while the smallest 16 funders provide only 1%. Many respondents noted that since 2000, there are not just more funders, but that there are more larger funders: instead of two major funders and a host of smaller ones, there is now a much larger number of more equal partners in the funding community.

Fig. 1. Sources of funding for cancer research in Canada, 2007. Data from CCRA report (ref 1)

Other funding for cancer research

Direct provincial investment

CCRA's remarkable compilation of Canada's cancer research granting activities⁷ is an essential foundation for starting to build a co-ordinated national research agenda. However, the cancer research scene is extraordinarily complex, and CCRA acknowledges that its data are not complete. In particular, it does not include the much harder-to-define funding which is provided directly to a specific institution rather than being widely distributed through a peer review mechanism. There are two major sources of this direct research funding, both of which are growing at extraordinary rates: cancer charities and provincial agencies. These investments cannot be directly compared, or added to, the CCRA's total of $400M in grants, as they capture much funding for research infrastructure and other support which is not usually paid for through grants. In addition, there is some overlap, as the same funder may provide some of their funding through peer review mechanisms, and some as direct investment in institutions.

Of the major new players, the most oft-mentioned by respondents is the Ontario Institute of Cancer Research (OICR). With an annual budget of about $80M, this organization alone has shifted the nature of Canadian research. Other provinces are also now making major investments in cancer research. British Columbia undertakes research within the BC Cancer Agency, and with funds raised by the BC Cancer Foundation. Alberta is supporting a mix of grant and direct research in academic and intramural institutions through the new Alberta Cancer Research Institute and the Cancer Legacy funds, with additional fund-raising support from the Alberta Cancer Foundation. Collectively, the provinces and federal government have come together under the umbrella of the Canadian Partnership against Cancer (CPAC), whose cancer control strategy includes a new investment in research of about $50M over the 5 year current lifespan of the strategy.

The provinces as health care providers and funders have different priorities than those of academically-oriented funders like CIHR and the former NCIC, and are rapidly shifting the focus of research towards translational activities and health care improvement. The research they fund is more likely to take place in hospital and community health care facilities than in academia, rendering the distinction between investments in research and investments in practice less clear: the health care infrastructure supports research activity, just as the research activity supports enhancements in care. For example, the BC Cancer Foundation reports providing the BC Cancer Agency with almost $20M in 2005-06 for "core research," and another $13M which supports both research and clinical enhancements.⁷ Provincial funding is also much more targeted, often quite specifically directed, and increasingly, as is the case with CPAC, contractual rather than grant-based. The provinces and CPAC are also making massive investments in the types of science funding that Canadian granting agencies are traditionally ill-equipped to make, such as large-scale projects and major platforms, which entail long-term commitments.

Direct charitable investment

In the charitable sector, "there's been an explosion in the number of groups that are raising funds and advocating – it's a bit dizzying at times!" Many charities invest funds as block support to a specific research institute, rather than distributing it widely through individual peer-reviewed grants. For example, the "Week-End to End Breast Cancer" raised $33M in 2009, most directly invested in specific research institutions across Canada; the Toronto event alone raised $17M, provided to the Princess Margaret Hospital. A linked "Ride to Conquer Cancer" raised an additional $14M over a one-day Toronto event. The Canadian Breast Cancer Foundation's "CIBC Run for the Cure" raised $26.5M in 2009, used for both peer reviewed funding through the Canadian Breast Cancer Research Alliance, and direct investment.

Funds raised through public donations to charities and foundations also tend to be quite targeted, with a focus on those areas which have captured public attention. As a result, charity funding can become highly concentrated in a few areas; breast cancer fundraising has in particular become a major national endeavour. The CCRA data show that research funding for this type of cancer, as well as for cancers that affect children, is higher, proportional to health impact, than "below the belt" cancers, such as prostate, bladder or colorectal.

CIHR's cancer research investment

CIHR's investment can be measured in a number of ways (Fig 2),⁸ but all show a significant increase since 2000, though commensurate with CIHR's overall budget growth, since the percentage of the research spend devoted to cancer research has remained constant. CIHR's cancer investment remains overwhelmingly for biomedical research (Fig. 3), though investments in other themes of health research are growing faster than the biomedical theme.

The thematic distribution of the Institute's strategic research funds is dramatically different from CIHR's overall profile, and has consistently been more evenly distributed between the four themes. This difference has resulted from the Institute's choice, as discussed in later sections, to invest in supporting under-funded and under-investigated areas of cancer research.

The distribution of CIHR and other funders' investments by funding mechanism (grants, salary and training awards, etc.), for different research aims (etiology, prevention, treatment, etc.) and for different sites or types of cancer (for all federal funding, of which CIHR accounts for 55%) are comprehensively described in the CCRA report.

Fig. 2. CIHR and ICR investment in cancer research (see ref. 8 for explanation of legend)

Fig. 3. CIHR investment in cancer research, allocated by the four CIHR themes of health research. The ICR 2008-09 data include contributions from other CIHR Institutes to ICR funding initiatives.

The Institute's research budget

While nobody knows for sure what the annual non-industrial Canadian investment in cancer research is, it is certain that it greatly exceeds the $400M reported by CCRA for 2005-06, perhaps by as much as $100M-$200M, and seems to still be growing. It's no surprise that respondents overwhelmingly agreed that transformation of the funding scene is the most significant change in the Canadian research environment over the last ten years.

When it comes to defining ICR's place in that environment however, there is a divide between insiders, who see ICR as the largest funder of cancer research in Canada, and outsiders, who see it as only a small player. Major stakeholders do not consider the larger CIHR spending on cancer research as pertinent to the Institute, given the Institute's lack of influence over how those funds are spent. To them, the Institute is a $10M enterprise – in other words, a bit player in the Canadian cancer scene. Many stakeholders felt the Institute's small budget severely undermines its ability to be effective: "The potential for impact can't be separated from the small budget."

A number of respondents particularly mentioned CIHR's failure to expand the role and budgets of the Institutes, as recommended by the International Review Panel at CIHR's fifth-year evaluation:⁹ "If the Institute had some more control over the $120M (that CIHR invests in cancer research), it would be a big difference, even if it was still all invested in investigator-initiated projects. I wonder what is the right emphasis of Institutes in CIHR?" In CIHR's initial years it was reasonable to reserve most of the CIHR grants budget centrally, since the Institutes were an unproven concept. After ten years of operation, given the endorsement the Institutes received from the International Review, as well as the many examples of successful and innovative Institute activities in the five years since, it would be expected that the Institutes collectively would be responsible for much more than roughly 15% of the CIHR budget. However, the Institutes do not appear to have become, in practice, the organizational structure of the agency, nor is this issue addressed in CIHR's new strategic plan.

For most of the CIHR Institutes, their budgets, while limited, were nonetheless enough to make them a notable player and leader in their area. But into this incredibly rich tapestry of cancer research funding, the ~$8M strategic research budget of the CIHR Institute of Cancer Research is barely noticeable (Fig.1). This has posed a major challenge to the Institute: with a budget that is insignificant relative to such a richly-funded area of research, how can it have a meaningful impact?

Investments in infrastructure vs. operating grants

Institutions across Canada have promoted cancer as a research priority, and cancer researchers have been very successful in competing for federal investments through the Canada Foundation for Innovation (CFI), Canada Research Chairs (CRCs) and Genome Canada. For example, some 230 Canada Research Chairs (out of the total of 700 allocated to health research) work in areas relevant to cancer, while CFI and its partners have provided about $850 million in new infrastructure relevant to cancer research over the past 12 years, out of a total of about $12B across all fields of research from archaeology to zoology. There has been immense growth in the size of the cancer research community: a quick impression of this growth is given by Table 1, which also speaks to productivity growth.

Table 1 (Web of Science data) Number of authors* on Canadian cancer publications with:	2000	2008
10 or more publications in the year	15	138
6 or more publications in the year	68	435
Any publications in the year	5,824	16,187

* this includes foreign collaborators

With respect to the CIHR-funded population, 779 names were associated with CIHR grants in the area of cancer in 2000-01, and 1403 in 2008-09.

While these new investments have brought extraordinary benefits to Canadian science, CIHR and the research community as a whole have been concerned for years about the imbalance between investment in research infrastructure and personnel versus operating costs.¹⁰ This imbalance is perceived as a particularly significant problem in the cancer area, exacerbated by the massive investments of the Ontario Institute of Cancer Research and others: "As OICR continues bringing in more people and infrastructure, the problem is in the number of people applying for CIHR grants. This increase is not compensated for by funding for operating grants."

Although this concern was frequently raised by respondents, we found no evidence that cancer researchers are being selectively disadvantaged in CIHR open grants competitions (Appendix B): success rates have plummeted in all areas. In addition, there is little evidence that application pressure has increased disproportionately for cancer-related applications. Nonetheless, our analysis was partially compromised by missing data, and the Institute may wish to undertake a more thorough analysis. In addition, success rates among other funders are also decreasing: NCIC success rates were around 37% in the early 2000s, but have recently fallen below 20%.¹¹

New funding needs: supporting "big science"

One of the biggest changes noted in the Canadian research environment over the last ten years is the growth of "big science," including genomics, stem cells, cohorts, international consortia, and mega-projects. This shifting research approach demands commensurate shifts in research funding approaches.

Several respondents suggested that newcomers like OICR and CPAC excel in creating and supporting such platforms, but that traditional funders like CIHR, focused on operating grants, aren't equipped to deal with this kind of project. Some respondents even wondered if CIHR will become marginalized, focusing on cancer research approaches which are becoming obsolete: "Individual scientists, 1-2 people in small labs doing bits of bench work – this is an idea of the past." Others, not surprisingly, worry that far too much funding and emphasis is going to mega- projects and strategic research, at the expense of individual operating grants.

In addition to funding the mega-projects themselves, funders need to consider the downstream implications of the creation of these platforms. What kinds of follow-on research and translational capacity which will be needed to "turn massive data-producing activities like genomics into implementable health care improvements"? Furthermore, as platforms such as high-throughput genome sequencing and large cohorts come together, "there's a melding of hitherto very remote disciplines," with enormous potential to advance areas such as personalized medicine: "Cancer has a greater chance of moving there than others. We're undergoing a revolution across the world."

Funders thus need to consider not only what they will fund, but just as importantly, how they will fund it. "Big science" requires more oversight from funding agencies than merely up-front peer review. CIHR has attempted to transpose its traditional operating grant programs, review and "hands-off" management processes onto strategic investments, and many respondents believe this is not working well. Strategic investments require an active and ongoing partnership between the researcher and the funding agency, and active management of the investment. They involve assembling and maintaining funding consortia, and making funding choices based on probable impact as well as science excellence, given that relatively few such large initiatives can be funded. While the Institutes would seem ideally placed to perform this role for CIHR, in practice the vast majority of the CIHR budget remains outside the purview of the Institutes.

Changes in Canadian cancer research performance

Canadian cancer research productivity

With Canadian cancer research funding quadrupling over the decade and continuing to grow, we would expect to see a major effect on research output, as measured in terms of quantity and hopefully quality of scientific publication. Canadian publications in cancer research have in fact increased steadily in number over the past dozen years, with two phases evident¹² (Fig. 4). The first, from 1998 to 2002 was characterised by a slow rate of growth, about 5% each year. From 2003 onwards, the rate of growth has been more variable, but has averaged 12% per year. This accelerated growth in Canadian publications follows, with an appropriate time-lag between receiving funding and publishing the results, the increased federal investment in health research that began in the late 1990s, including the formation of CIHR and the tripling of its budget, and the renewal of research infrastructure through the Canada Foundation for Innovation. Thus the quadrupling of funding has resulted in a roughly 2.5-fold increase in publication number.

Fig. 4. Canadian cancer publications/year (columns) and world share as % (line).
2009 records were probably incomplete at the time they were accessed (March 2010).

Canada's rising funding has taken place in a context of rapidly growing global cancer research investments: the budget of the US National Cancer Institute, for example, almost doubled during this period (from US$2.5 billion to US$4.8 billion).¹³ Canada's share of global cancer publications thus fluctuated between 6th and 8th place over the period 1999-2008 (Fig. 5).¹⁴ Nevertheless, Canada's share of publications has increased from 3% to almost 4% (Canada's overall average in all fields of research) over the past decade, an impressive achievement given that with the growth of new leaders like China and South Korea, many other major players such as the US, UK, Japan, and Germany actually saw their share decline during this same period.

Fig. 5. Leading nation share of world cancer research publications

Cancer research specialization

Fig. 6 compares Canadian cancer publications to the world output, to identify areas where Canada produces a proportionately larger volume of cancer research publications.¹⁵ There is striking Canadian specialization, relative to the world, in non-biomedical aspects of cancer research such as social sciences; health services and policy; public, environmental and occupational health; and psychology. Paediatrics and haematology are also significant areas of Canadian cancer research strength. Canada is under-specialized in surgery and its sub-specialities (except for urology, possibly related to special funding for prostate cancer research); in some of the disciplines and approaches that bear directly on commercialization (biotechnology; chemistry; toxicology; pharmacology; biochemical research methods) and on the physical sciences (chemistry; physical and material sciences and engineering). For the largest fields of cancer research (e.g. oncology, cell biology, biochemistry, general and internal medicine), Canada's research output is proportional to the global output.

Fig. 6. Index of specialization, Canadian cancer research, 2008

International collaboration

Like many areas of health research, cancer research in Canada is becoming increasingly international (Fig. 7). The percentage of Canadian papers with a foreign author¹⁶ has increased for every country, doubling on average, and highest with Spain, China and Germany. The overall percentage of Canadian publications resulting from an international collaboration has increased, and their importance is clear: cancer research publications from international collaborations are cited on average about twice as often as publications with only Canadian authors.

Table 1. International collaboration

Published in	1998-2000	2006-2008
% Canadian publications with a foreign collaborator	35.3%	42.2%
citations per publication, all authors Canadian	28.4	5.8
citations per publication, with foreign co-authors	57.4	12.4
Note: that citations for 2006-8 publications are lower simply because less time has elapsed since these publications appeared: they say nothing about quality differences between older and newer publications.

Fig. 7. Countries with which Canadians collaborate: % of Canadian cancer research publications with co-authors from other leading nations

While international collaboration increased dramatically in all countries over the period 2001-2008, Canadian authors in the field of oncology¹⁷ are consistently among the most likely to collaborate internationally (Fig. 8), except for China, where an extraordinary but transient phase of international collaboration began somewhat earlier, as that nation's research enterprise developed rapidly. Researchers from the USA are consistently less likely to collaborate internationally.¹⁸

Canadian research quality

Canada's cancer research publications attract a great deal more than average attention, suggesting they have high quality and/ or importance. Given that Canada's share of world publications in cancer research has risen from 3% to 4%, it would be reasonable to expect to find 3 or 4 Canadian papers among the world's 100 most-cited papers. Instead, Canada produced 14 of them in 1998-2000, and 13 in 2006-2008.¹⁹

Similarly, while Canada has ranked 6th-8th in terms of its share of global publications, it has consistently ranked 3rd or 4th in terms of citation frequency²⁰ (Fig. 9), showing that its publications are high impact, and the major growth in their number has not reduced their quality.

Fig. 8. Collaborativeness by country in oncology research: % of publications with a foreign author.

Fig. 9. Ranking of the leading cancer research nations in order of the average number of citations to their publications from the years indicated