[EQ] Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials

Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials

Laura Haynes, Visiting Researcher at King’s College London

Owain Service, Deputy Director of the Behavioural Insights Team
Ben Goldacre,  Research Fellow at London School of Hygiene and Tropical Medicine

Professor David Torgerson, Director of the York Trials Unit

Published by UK Cabinet Office Behavioural Insights Team – June 2012

 

Available online at: http://bit.ly/NGBz3Q

“……Randomised controlled trials (RCTs) are the best way of determining whether a policy is working. They are now used extensively in international development, medicine, and business to identify which policy, drug or sales method is most effective. They are also at the heart of the Behavioural Insights Team’s methodology.

However, RCTs are not routinely used to test the effectiveness of public policy interventions in the UK. We think that they should be.

What makes RCTs different from other types of evaluation is the introduction of a randomly assigned control group, which enables you to compare the effectiveness of a new intervention against what would have happened if you had changed nothing.

The introduction of a control group eliminates a whole host of biases that normally complicate the evaluation process – for example, if you introduce a new “back to work” scheme, how will you know whether those receiving the extra support might not have found a job anyway?....”

Content

Executive Summary

Introduction

Part 1 -What is an RCT and why are they important?

What is a randomised controlled trial?

The case for RCTs - debunking some myths

1.We don’t necessarily know‘what works’

2. RCTs don’t have to cost a lot of money

3. There are ethical advantages to using RCTs

4. RCTs do not have to be complicated or difficult to run

PART II - Conducting an RCT: 9 key steps

Test

Step1: Identify two or more policy interventions to compare

Step 2:Define the outcome that the policy is intended to influence

Step 3:Decide on the randomisation unit

Step 4:Determine how many units are required for robust results

Step 5: Assign each unit to one of the policy interventions using a robustly random method

Step 6: Introduce the policy interventions to the assigned groups

Learn

Step 7: Measure the results and determine the impact of the policy interventions

Adapt

Step 8: Adapt your policy intervention to reflect your findings

Step 9: Return to step 1

KMC/2012/SDE
Twitter http://twitter.com/eqpaho

 *      *     *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues.  [DD/ KMC Area]
Washington DC USA

“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
------------------------------------------------------------------------------------
PAHO/WHO Website
Equity List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html
Twitter http://twitter.com/eqpaho

IMPORTANT: This transmission is for use by the intended
recipient and it may contain privileged, proprietary or
confidential information. If you are not the intended
recipient or a person responsible for delivering this
transmission to the intended recipient, you may not
disclose, copy or distribute this transmission or take
any action in reliance on it. If you received this transmission
in error, please dispose of and delete this transmission.

Thank you.