This is a post by Amanda Glassman reproduced with permission of the Center for Global Development

Cost-effectiveness studies compare the costs and benefits of different interventions with the aim of improving decisions on the allocation of scarce resources for health.  Or, put simply, they allow policy-makers to set priorities for health spending and consider how the next dollar available can get more health for the money. 

The Bill & Melinda Gates Foundation (BMGF) funds about a fifth of all published cost-effectiveness studies on interventions to address AIDS, tuberculosis, malaria, and vaccine-preventable diseases in low- and middle-income countries. BMGF also plays a high-profile role in promoting the concept of cost effectiveness as a criterion for global health decision making and spending.

But for cost-effectiveness studies to actually improve decisions, methods must be appropriate and reporting must be clear and accurate. If not done well, these analyses can be difficult to interpret and can lead to suboptimal or even incorrect decisions.

So BMGF recently commissioned NICE International, the University of York, and the Health Intervention and Technology Appraisal Program (Thailand) to develop the Gates Reference Case, a principle-based standardized methodology for economic evaluation in developing countries. The principles are described in detail here, and cover issues of transparency, comparators, better use of evidence, and measures of outcomes, among others.  Using these principles, they also assess retrospectively how published BMGF-funded studies have fared since 2000.

The results of the study were disappointing.  Paraphrasing the report:

“Most studies provided insufficient information about currency conversions and/or methods for adjusting costs to account for temporal disparities. Where information was provided, crude exchange rates were frequently used to convert unit costs drawn from other settings (often high-income countries). There was poor adherence to the three key methodological specifications for DALY estimation, raising significant concerns as variant approaches to DALY calculation limit comparability between studies. Although widely considered as the most comprehensive method of dealing with the various sources of uncertainty in economic evaluations, few studies presented probabilistic sensitivity analyses. Generalizability and transferability of results and equity implications of evaluated interventions were discussed in less than one-third of all reviewed studies. Only 35% of studies discussed the affordability of the interventions being assessed, despite these studies being undertaken in very resource-limited settings.”

Bottom line: it is tough to use this body of evidence to make better decisions.

Reference cases have been in the public domain for some time, and have been adopted by the US Panel on Cost-Effectiveness in Health and Medicine, the World Health Organization, and NICE itself, as a means to improve quality and comparability in the conduct and reporting of cost-effectiveness analyses. BMGF adoption could greatly improve the quality of economic evaluation for global health, particularly if the case is used as a condition for funding and a criterion for a specialized peer review as part of the commissioning and oversight of cost-effectiveness studies.

At the recent launch of the Reference Case, the Foundation announced plans to create incentives for researchers to adhere more closely to the best practice principles laid out in the Reference Case. Other cost-effectiveness analysis funders should follow suit; Wellcome Trust/MRC, DFID, USAID, and others could use the same standards and even the same peer review mechanism. Widespread use could enable more meaningful and explicit comparison of analyses and findings across multiple studies, which in turn will allow cost-effectiveness analyses to better guide health care spending decisions in developing countries.