Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials
Laura Haynes, Visiting Researcher at King’s College London
Owain Service, Deputy Director of the Behavioural Insights Team
Ben Goldacre, Research Fellow at London School of Hygiene and Tropical Medicine
Professor David Torgerson, Director of the
Available online at: http://bit.ly/NGBz3Q
“……Randomised controlled trials (RCTs) are the best way of determining whether a policy is working. They are now used extensively in international development, medicine, and business to identify which policy, drug or sales method is most effective. They are also at the heart of the Behavioural Insights Team’s methodology.
However, RCTs are not routinely used to test the effectiveness of public policy interventions in the
What makes RCTs different from other types of evaluation is the introduction of a randomly assigned control group, which enables you to compare the effectiveness of a new intervention against what would have happened if you had changed nothing.
The introduction of a control group eliminates a whole host of biases that normally complicate the evaluation process – for example, if you introduce a new “back to work” scheme, how will you know whether those receiving the extra support might not have found a job anyway?....”
Part 1 -What is an RCT and why are they important?
What is a randomised controlled trial?
The case for RCTs - debunking some myths
1.We don’t necessarily know‘what works’
2. RCTs don’t have to cost a lot of money
3. There are ethical advantages to using RCTs
4. RCTs do not have to be complicated or difficult to run
PART II - Conducting an RCT: 9 key steps
Step1: Identify two or more policy interventions to compare
Step 2:Define the outcome that the policy is intended to influence
Step 3:Decide on the randomisation unit
Step 4:Determine how many units are required for robust results
Step 5: Assign each unit to one of the policy interventions using a robustly random method
Step 6: Introduce the policy interventions to the assigned groups
Step 7: Measure the results and determine the impact of the policy interventions
Step 8: Adapt your policy intervention to reflect your findings
Step 9: Return to step 1
* * *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues. [DD/ KMC Area]
“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
Equity List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html
IMPORTANT: This transmission is for use by the intended
recipient and it may contain privileged, proprietary or
confidential information. If you are not the intended
recipient or a person responsible for delivering this
transmission to the intended recipient, you may not
disclose, copy or distribute this transmission or take
any action in reliance on it. If you received this transmission
in error, please dispose of and delete this transmission.