Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go?
David Ogilvie, Matt Egan, Val Hamilton, Mark Petticrew
J Epidemiol Community Health 2005;59:886–892. doi: 10.1136/jech.2005.034199
http://jech.bmj.com/cgi/reprint/59/10/886?maxtoshow=&HITS=10&hits=10&RESULTFORMAT=&author1=Petticrew&andorexactfulltext=and&searchid=1&FIRSTINDEX=0&sortspec=relevance&resourcetype=HWCIT
Study objective: There is little guidance on how to select the best available evidence of health effects of social interventions. The aim of this paper was to assess the implications of setting particular inclusion criteria for evidence synthesis.
Design: Analysis of all relevant studies for one systematic review, followed by sensitivity analysis of the effects of selecting studies based on a two dimensional hierarchy of study design and study population.
Setting: Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.
Main results: The distribution of available evidence was skewed. Population level interventions were less likely than individual level interventions to have been studied using the most rigorous study designs; nearly all of the population level evidence would have been missed if only randomised controlled trials had been
included. Examining the studies that were excluded did not change the overall conclusions about effectiveness, but did identify additional categories of intervention such as health walks and parking charges that merit further research, and provided evidence to challenge assumptions about the actual
effects of progressive urban transport policies.
Conclusions: Unthinking adherence to a hierarchy of study design as a means of selecting studies may reduce the value of evidence synthesis and reinforce an ‘‘inverse evidence law’’ whereby the least is known about the effects of interventions most likely to influence whole populations. Producing generalisable estimates of effect sizes is only one possible objective of evidence synthesis. Mapping the available evidence and uncertainty about effects may also be important.
Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go?
David Ogilvie, Val Hamilton, Matt Egan, Mark Petticrew
J Epidemiol Community Health 2005;59:804–808. doi: 10.1136/jech.2005.034181
Study objective: There is little guidance on how to identify useful evidence about the health effects of social interventions. The aim of this study was to assess the value of different ways of finding this type of information.
Design: Retrospective analysis of the sources of studies for one systematic review. Setting: Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.
Main results: Only four of the 69 relevant studies were found in a ‘‘first-line’’ health database such as Medline. About half of all relevant studies were found through the specialist Transport database. Nine relevant studies were found through purposive internet searches and seven relevant studies were found by chance. The unique contribution of experts was not to identify additional studies, but to provide more information about those already found in the literature.
Conclusions: Most of the evidence needed for this review was not found in studies indexed in familiar literature databases. Applying a sensitive search strategy across multiple databases and interfaces is very labour intensive. Retrospective analysis suggests that a more efficient method might have been to search a
few key resources, then to ask authors and experts directly for the most robust reports of studies identified. However, internet publications and serendipitous discoveries did make a significant contribution to the total set of relevant evidence. Undertaking a comprehensive search may provide unique evidence and insights that would not be obtained using a more focused search.
Evidence for public health policy on inequalities: 2: Assembling the evidence jigsaw
Margaret Whitehead, Mark Petticrew, Hilary Graham, Sally J Macintyre, Clare Bambra, Matt Egan
J Epidemiol Community Health 2004;58:817–821. doi: 10.1136/jech.2003.015297
Study objective: To garner research leaders’ perceptions and experiences of the types of evidence that influence policy on health inequalities, and their reflections on how the flow of such research evidence could be increased.
Design, setting, and participants: Qualitative two day residential workshop with senior research leaders, most of whom were currently involved in evaluations of the health effects of major policies. In four in depth sessions, facilitated by the authors in turn, focused questions were presented to participants to reveal their views and experiences concerning evidence synthesis for policy on inequalities. These were analysed thematically.
Main results: Five types of evidence for policy on health inequalities were felt to be particularly persuasive with policymakers: observational evidence showing the existing of a problem; narrative accounts of the impacts of policies from the household perspective; controlled evaluations; natural policy experiments; and historical evidence. Methods of improving the availability and use of these sources of information were put forward.
Conclusions: This paper and its companion have considered the current evidence base for policies to reduce health inequalities, and how this could be improved. There is striking congruence between the views of the researchers in this study and policy advisers in paper 1, suggesting that a common understanding may be emerging. The findings suggest significant potential for rapid progress to be made in developing both evidence based policy, and policy relevant evidence to tackle inequalities in health.
Evidence for public health policy on inequalities: 1: The reality according to policymakers
Mark Petticrew, Margaret Whitehead, Sally J Macintyre, Hilary Graham, Matt Egan
J Epidemiol Community Health 2004;58:811–816. doi: 10.1136/jech.2003.015289
http://jech.bmj.com/cgi/reprint/58/10/811?ck=nck
Objective: To explore with UK and international policy advisors how research evidence influences public health policy making, and how its relevance and utility could be improved, with specific reference to the evidence on the production and reduction of health inequalities.
Design, setting, and participants: Qualitative residential workshop involving senior policy advisors with a substantive role in policy development across a range of sectors (mainly public health, but also including education, social welfare, and health services). In four in depth sessions, facilitated by the authors, focused questions were presented to participants. Their responses were then analysed thematically to identify key themes, relating to the availability and utility of existing evidence on health inequalities.
Main results: The lack of an equity dimension in much aetiological and evaluative research was highlighted by participants. Much public health research was also felt to have weak underlying theoretical underpinnings. As well as evaluations of the effectiveness and cost-effectiveness of policy and other interventions, they identified a need for predictive research, and for methodological research to further develop methods for assessing the impact on health of clusters of interventions.
Conclusions: This study reinforces the view that there is a lack of information on the effectiveness and cost effectiveness of policies, and it uncovered additional gaps in the health inequalities evidence base. A companion paper discusses researchers’ views of how the production of more relevant public health evidence can be stimulated.
* * * *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues. [DD/ KMS Area]
“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
------------------------------------------------------------------------------------
PAHO/WHO Website: http://www.paho.org/
EQUITY List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html
