Research Profile – Context is Key

A research team at the University of British Columbia is helping policy makers evaluate the success of new health care policies in different settings.

[ Back to main article ]

When you set out to fix complex problems, like inefficiencies in a health care system, there are no one-size-fits-all solutions. A new approach to hand hygiene that successfully reduces infection rates in one hospital may not work in another. So how can policy makers determine which health care interventions will succeed and which will fail?

Dr. Allan Best, at the University of British Columbia, helps answer these difficult questions.

"Context accounts for a huge amount of variance in the implementation of most evidence-based interventions, and we need to understand how that local context is interacting with the intervention to produce the results," explains Dr. Best.

To do so, Dr. Best and his colleagues use a technique called the Rapid Realist review.

Systematic reviews, which synthesize all of the research on a particular question, are powerful tools. But they can be time-consuming to conduct and difficult for policy makers to interpret. And sometimes there isn't enough existing data, particularly quantitative research, to draw upon.

But a Rapid Realist review includes many types of evidence: quantitative data, such as the results of randomized controlled trials, theory, narrative analysis and practice knowledge.

"So we end up with a richer combination of research, theory, and practice evidence that allows us to say more than we would from a narrower, more quantitative kind of review," says Dr. Best.

Each Rapid Realist review team includes an expert panel and a reference panel. The expert panel is usually composed of researchers who are very familiar with the studies and theories relevant to the intervention and the different methods for analyzing those studies. These experts help speed up the review process by reducing the amount of time needed to gather and analyze the research. They also provide critical practice-based knowledge that can help fill gaps that the literature has not yet been able to address.

The reference panel, on the other hand, represents people who will actually use the results of the research and who keenly understand the environment in which they will be used.

"We want a reference group that represents the various players that will need to have some buy-in if [the review results are] going to be taken up and used," says Dr. Best.

With the help of the Canadian Institutes of Health Research, Dr. Best and his team are using this technique to help the Saskatchewan Ministry of Health assess the implementation of "lean" management. Popularized by the Toyota Motor Corporation, lean management emphasizes the customer's needs (or in this case, the patient's and family's needs) and tries to streamline and enrich the delivery process.

Over the last few years, Saskatchewan has introduced this management approach in more than 150 demonstration projects across the province and is in the process of expanding it throughout the province's health care system.

As part of this transformation effort, the Ministry of Health needs to determine how the changes to a lean management culture can be sustained.

"Because we're operating in a complex, adaptive system, you're not going to get the clean-cut application that you would with a manufacturing line," says Dr. Best. "But the same principles should apply."

The Rapid Realist review that Dr. Best and his team are currently conducting for Saskatchewan will help the province create an evaluation system that will determine whether leadership and management culture have really transformed, and to what degree. It will also point to key "pressure points" that the Saskatchewan government can address to increase the chance that the changes will be sustained.

"Our experience with Dr. Best and his team is a great demonstration of the power of collaboration between the policy and research communities," says Pauline Rousseau, Executive Director of the Strategy and Innovation Branch at the Ministry of Health.

The Ministry of Health first worked with Dr. Best in 2010 on a knowledge synthesis on large system transformation.

"We are delighted to have another opportunity to work with Dr. Best and his team on a new review and synthesis focused on transforming management culture in large systems," says Ms. Rousseau. "The timing couldn't be better as the province embarks on a significant expansion of our efforts to transform the patient experience, outcomes and cost of care through the system-wide adoption of lean."

"What a realist review is trying to answer is the question, 'What worked in what circumstances, and why?'."
– Dr. Allan Best, University of British Columbia