[EQ] Methodological problems in the use of indirect comparisons for evaluating healthcare interventions

Methodological problems in the use of indirect comparisons for evaluating healthcare interventions:
survey of published systematic reviews

 

Fujian Song, reader in research synthesis,1 Yoon K Loke, senior lecturer in clinical pharmacology,1 Tanya Walsh, lecturer in dental statistics,2 Anne-Marie Glenny, lecturer in evidence based oral care,2 Alison J Eastwood, senior research fellow,3 Douglas G Altman, professor and director4

1 Faculty of Health, University of East Anglia, Norwich

2  School of Dentistry, University of Manchester, Manchester

3  Centre for Reviews and Dissemination, University of York, York

4  Centre for Statistics in Medicine, Oxford

BMJ April 18, 2009;338:b1147 - doi:10.1136/bmj.b1147

 

Available online at: http://www.bmj.com/cgi/reprint/338/apr03_1/b1147

 

Abstract: http://www.bmj.com/cgi/content/abstract/338/apr03_1/b1147

 

“……The number of available healthcare interventions increases with time, reflecting advances in science and technology. For many clinical indications clinicians may have to choose among several competing interventions. In this era of evidence based decision making, relative effectiveness and cost effectiveness of different interventions need to be objectively and accurately assessed in clinical studies.

 

It has been accepted generally that well designed and implemented head to head randomised controlled trials provide the most rigorous and valid research evidence on the relative effects of different interventions.1 Evidence from head to head comparison trials is often limited or unavailable, however, and indirect comparison may therefore be necessary…..”

 

ABSTRACT

Objective To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions. Design Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used.

Data extraction Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned.

Results The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases.

Conclusions Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems.

Appendix 1 PubMed search strategy

Appendix 2 Characteristics of identified reports

Appendix 3 Identified studies

References of included studies

 

 

*      *      *     * 
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues.  [DD/ KMC Area] 

“Materials provided in this electronic list are provided "as is".Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
---------------------------------------------------------------------------------------------------

PAHO/WHO Website: http://66.101.212.219/equity/  

EQUITY List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html

 

 

 

    IMPORTANT: This transmission is for use by the intended recipient and it may contain privileged, proprietary or confidential information. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you may not disclose, copy or distribute this transmission or take any action in reliance on it. If you received this transmission in error, please notify us immediately by email to infosec@paho.org, and please dispose of and delete this transmission. Thank you.