Cochrane Corner Column

Sensitizing Systematic Reviews for Sex/Gender, Equity and Bias: Some Challenges

Vivien Runnels, Marion Doull, Sari Tudiver and Madeline Boscoe

Systematic reviews are designed to evaluate the effectiveness of interventions in health and policy by gathering and analyzing evidence from multiple primary studies. Patients, practitioners and others consult systematic reviews for evidence-based recommendations that are applicable and context specific. However, systematic reviewers encounter challenges in acquiring and synthesizing data in relation to sex and gender, which in turn have implications for methods bias and equity in outcomes. We report here on some of these challenges synthesized from responses to a questionnaire developed by our Working Group on Sex and Gender in Systematic Reviews that was circulated to colleagues with interest and expertise in systematic reviews prior to the meeting "Combining Forces to Improve Systematic Reviews: Gender, Equity and Bias" (Ottawa, ON, May 2011).

Conceptual challenges
How a problem is framed, conceptualized and ultimately researched is determined by decisions made early in the systematic review process. Upstream decisions shape the production of knowledge including determining when to investigate sex and gender differences, when to factor in questions of equity and how to address potential issues of bias in trial design. However, these decisions can only be made on the basis of pre-existing knowledge. Conceptual challenges include lack of understanding the distinct, but interrelated concepts of sex and gender, lack of awareness about potential sex/gender-based differences and the importance of equity for health interventions, research and outcomes.

Methodological challenges
Assessing sex, gender, equity and bias in systematic reviews presents numerous methodological challenges. These include: poor quality protocols; lack of transparency of clinical trials and the resultant inability to determine whether outcomes and analyses were specified a priori; difficulties in completing robust subgroup analysis when there is under-representation of those from disadvantaged groups in the trial population and when published studies do not provide sex-disaggregated data. These limitations suggest a need for appropriate guidance and training for researchers, systematic reviewers and the broader research community (e.g. journal editors, publishers, funders).

From concepts to methods
Overall, colleagues emphasized that a lack of conceptual clarity is interconnected to, and iteratively feeds, methodological challenges. This laid the basis for discussions at the "Combining Forces" meeting where we identified a need for indicators for gender and other health determinants, and for guidelines for how to achieve quality sex/gender analysis within reviews and for theorizing how to interpret generalized data for diverse individuals and groups. Together, we pointed to the value of highlighting gaps in evidence even if appropriate sub-group analyses cannot be carried out and noted that a lack of tools, checklists, and/or outlines for conducting sex/gender analysis in systematic reviews creates a challenge for methodologists unfamiliar with the concepts of sex and gender in particular.

The conceptual and associated methodological challenges that the inclusion of sex, gender, equity and bias can present for systematic reviewers have limited the uptake of these issues and their associated analyses into reviews. Addressing the challenges by developing methods and tools to facilitate and build critical capacity has significant potential for enhancing the applicability and the quality of the evidence to achieve better health outcomes for all.

The authors thank the participants at the "Combining Forces" workshop whose thoughtful input allowed them to contribute this column. "Combining Forces" was funded through an IGH Meetings, Planning and Dissemination Grant.

Integrating Sex and Gender in Logic Models for Systematic Reviews

Erin Ueffing and Jordi Pardo Pardo

Logic models are visual representations of theories about how an intervention works in given contexts. Logic models have been suggested as tools to improve the understanding of whether an intervention works in a particular population: "Equity oriented systematic reviews should include a logic model to elucidate hypotheses for how the intervention (whether a policy or a programme) was expected to work, and how factors associated with disadvantage (social stratification) might interact with the hypothesized mechanisms of action".1

One example of a logic model used in systematic reviews and practice guidelines is that developed by the US Preventive Services Task Force.2 This logic model, or "analytic framework", illustrates the population, interventions, and outcomes to be considered in a systematic review. It can be adapted easily to map the influence of sex and gender on the various links between the population and the potential outcomes (Figure 1). For example, consider a community program for HIV/AIDS. Both sex and gender would be important considerations when identifying persons at risk: females are more susceptible to HIV than males, while women often have less sexual power or control than men. Sex and gender also play roles in risk groups such as commercial sex workers or men who have sex with men. For those who participate in screening programs, potential adverse effects vary by gender: women may be at higher risk than men for community exclusion, spousal violence or rejection, and family conflict both if they are screened and if they are diagnosed with HIV.3 Treatment or interventions, too, may vary by gender. For example, many HIV/AIDS education and counselling programs are tailored by gender. In addition, many interventions intended to reduce HIV transmission are tailored by sex, such as female condoms. Finally, the intermediate outcomes that are chosen for evaluation may differ by sex. For example, the reduction in mother-to-child transmission of HIV would be an appropriate outcome only for those interventions aimed at females who are or may become pregnant.

As this example has shown, logic models can be used usefully and appropriately to map the potential influences of gender and sex from the population to the outcomes of an intervention. In systematic reviews, logic models such as this can serve a variety of purposes. For example, mapping gender and sex may justify the decision to limit a review to one sex or gender, or justify subgroup analyses to examine the differential effects of an intervention across sex and gender.4

Logic models are powerful tools to illustrate how sex and gender play roles in complex interventions, and understand how sex and gender link with other factors to modify the effects of an intervention or the condition for which an intervention is intended.

References

  1. Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, Bonnefoy J, Morgan A, Doohan E, Kelly MP. Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ. 2010 Sep 13;341:c4739. doi: 10.1136/bmj.c4739.
  2. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D; Methods Work Group, Third US Preventive Services Task Force. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001 Apr;20(3 Suppl):21-35.
  3. United Nations Population Fund (UNFPA). HIV Prevention Now Programme Briefs No.4 - Addressing Gender Perspectives in HIV Prevention. 2002. Available at the UNFPA website.
  4. Anderson L, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using Logic Models to Capture Complexity in Systematic Reviews. In press.

Sex and Gender in the Cochrane Library: Looking Back, Moving Forward

Erin Ueffing and Jordi Pardo Pardo, Campbell and Cochrane Equity Methods Group

In the ten years since the inception of the Institute of Gender and Health, there have been increasing calls from policy makers, practitioners, and researchers for the integration of gender and sex in health research.1 At the same time, there has been a movement toward evidence-based health care; that is, using available research evidence to guide the decision making affecting health care practices and patient care.2 International groups such as the World Health Organization's Commission on Social Determinants of Health have identified systematic reviews as one form of evidence that meets these needs. The Cochrane Collaboration publishes systematic reviews of health interventions in The Cochrane Library.

Looking back over the past decade, three studies have investigated how reviews produced through the Cochrane Collaboration have considered the applicability of the research evidence assessed with regards to differences by sex or gender. Two of these surveys focused exclusively on sex or gender,3,4 while the third also assessed other sociodemographic factors such as race/ethnicity, socioeconomic status, and education.5

The first survey, conducted by Johnson et al., examined 30 Cochrane reviews on heart, hypertension, and peripheral vascular diseases in 2001. Of these reviews, only three considered whether gender played a role in the effectiveness of interventions. Of the 196 studies that recruited both men and women and were included in these reviews, 65 (33%) examined outcomes by gender. Of the 65 trials that performed a gender-based analysis, 13 (20%) reported significant differences in cardiovascular-related outcomes by gender.3

In their 2010 update of Johnson's study, Doull et al. examined 38 randomly selected reviews on heart, hypertension, and peripheral vascular diseases published since 2001. As with the Johnson study, Doull et al. found that differences between men and women were rarely considered; only two of the 38 reviews presented results disaggregated by sex or gender. The authors further noted that the terms "sex" and "gender" were used interchangeably, and that reviews often reported on the populations included in primary studies in terms of "% male", without describing the remaining participants.

Tugwell and colleagues examined how sex, gender, and a range of sociodemographic characteristics were accounted for in reviews on rheumatoid arthritis. The authors included all 14 reviews published between 2003 and 2008, which collectively reported on 147 primary studies. Five reviews (35.7% of the 14) and 131 primary studies (89.1% of the 147) reported the proportion of men and women included as participants at the start of the primary studies. Notwithstanding the share of primary studies reporting this information, Cochrane reviews on rheumatoid arthritis did not assess whether the effects of interventions for rheumatoid arthritis differed for men and women. The authors concluded that important differences might be missed and that systematic reviews can help to identify evidence gaps to guide future research.5

These surveys show that across different time periods and subject areas, sex and gender are not considered or reported consistently in Cochrane reviews related to heart, hypertension, and peripheral vascular diseases and rheumatoid arthritis. Future work is needed to determine how Cochrane reviews on interventions in other areas of health fare in accounting for sex and/or gender. Moreover, these studies demonstrate that there are gaps in understanding sex and gender, from the definition of the concepts to the implementation of the review methods to analyze sex and gender. Cochrane reviews have not been sufficiently sensitive to results from gender- and sex-based analyses and may have underreported relevant findings.

Moving forward to tackle these knowledge gaps, the Campbell and Cochrane Equity Methods Group develops methods for adequately addressing sex- and gender-based analysis in systematic reviews. These methods help to determine when and how sex and gender should be considered in Cochrane reviews, and ways to address them in the analysis. The Equity Methods Group also works with Canadian and international colleagues to build the evidence base on the differential effects of interventions on the basis of sex and gender.

References

  1. Sen, G. and Östlin, P. (2008) 'Gender inequity in health: why it exists and how we can change it', Global Public Health, 3:1, 1 - 12
  2. Dawes M, Summerskill W, Glasziou P et al. Sicily statement on evidence-based practice. BMC Med Educ 2005; 5: 1– 7.
  3. Johnson SM, Karvonen CA, Phelps CL, Nader S, Sanborn BM. Assessment of analysis by gender in the Cochrane reviews as related to treatment of cardiovascular disease. J Womens Health (Larchmt ) 2003;12(5):449-57. Available: PM:12869292.
  4. Doull M, Runnels VE, Tudiver S, Boscoe M. Appraising the evidence: applying sex- and gender-based analysis (SGBA) to Cochrane systematic reviews on cardiovascular diseases. J Womens Health (Larchmt ) 2010;19(5):997-1003. Available: PM:20384450.
  5. Tugwell P, Maxwell L, Welch V, Kristjansson E, Petticrew M, Wells G, et al. Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group? Arthritis Rheum 2008;59(11):1603-10. Available: PM:18975366.

Visit IGH Cochrane Corner.

Supplemental content (right column)