Wednesday, November 9, 2011

[EQ] The Magenta Book - Guidance for evaluation

The Magenta Book - Guidance for evaluation

HM Treasury UK, April 2011

Available online PDF [141p.] at:


“…….The Government is committed to improving central and local government efficiency and effectiveness, and in times of constrained public finances it is even more important to ensure that public funds are spent on activities that provide the greatest possible economic and social return. This requires that policy is based on reliable and robust evidence, and high quality evaluation is vital to this.


HM Treasury’s Green and Magenta Books together provide detailed guidelines, for policy makers and analysts, on how policies and projects should be assessed and reviewed. The two sets of guidance are complementary:
- the Green Book emphasising the economic principles that should be applied to both appraisal and evaluation, and
- the Magenta Book providing in-depth guidance on how evaluation should be designed and undertaken.


The risk of not evaluating, or of poor evaluation, is that policy makers are not aware if policies are ineffective or, worse still, result in overall perverse, adverse or costly outcomes. If there is no good evaluation evidence to demonstrate it, then we cannot be confident that taxpayers’ money is being properly spent, even where policies are in reality highly effective. The knowledge we gain from good evaluation can be used to increase policy effectiveness and is essential in informing the development of new policies to achieve the best results…..”


Chapter 1 Key issues in policy evaluation

What is evaluation and what benefits can it bring?

What factors affect how a policy should be evaluated?

How evaluation fits into the policy cycle

Chapter 2 Identifying the right evaluation for the policy

How was the policy delivered? Process evaluation

What difference did the policy make? Impact evaluation

Did the benefits justify the costs? Economic evaluation

Why did what happened occur?

What type of evaluation for the policy?

How do evaluation questions relate to the underlying “logic” of the intervention?

Factors affecting the choice of evaluation approach

Chapter 3 Building impact evaluation into policy design

Thinking about impact evaluation when designing the policy

The role of comparison groups in identifying the impact of a policy

Chapter 4 What practical issues need to be taken into account when designing an evaluation

The main steps in the evaluation process

How to ensure an evaluation meets the requirements: governance and quality control

Timing of the evaluation

What types of resources are likely to be needed?

What level of resource should be dedicated to the evaluation

Chapter 5 The stages of an evaluation

The steps involved in planning and undertaking an evaluation

Chapter 6 Setting out the evaluation framework

Theory-based evaluation

Reviewing the existing evidence

Systematic review

Rapid evidence assessment

Meta-evaluation and meta-analysis

Making sense of existing and new evidence: simulation modelling

Chapter 7 Data collection

What is monitoring data and how can it contribute to evaluation?

New data collection

Designing data collection tools

Ethical and data protection considerations

Chapter 8 Process evaluation, action research and case studies

Evaluation to understand the implementation and delivery of policy

Process evaluation

Action research

Case studies

Why undertake a process evaluation, action research or case study?

Research methods to support process evaluation, action research and case studies

Choosing research methods

Research methods

Chapter 9 Empirical impact evaluation

Introducing empirical impact evaluation

When are empirical approaches possible?

Designing policies for effective evaluation

Power of design

Strategies for analysing quasi experimental data

Thinking critically about the textbook techniques

“Constrained designs”

Chapter 10 Drawing together and reporting evaluation evidence

How evaluation evidence may be used

Drawing together the evaluation evidence

Setting the evaluation results in a broader context

Future decisions and roll-out; scaling-up

Implications for evaluation planning

Reporting and disseminating findings


You could also see:


The book divided into two parts.


Part A is designed for policy makers. It sets out what evaluation is, and what the benefits of good evaluation are. It explains in simple terms the requirements for good evaluation, and some straightforward steps that policy makers can take to make a good evaluation of their intervention more feasible. [28p]


Part B is more technical, and is aimed at analysts and interested policy makers. It discusses in more detail the key steps to follow when planning and undertaking an evaluation and how to answer evaluation research questions using different evaluation research designs. It also discusses approaches to the interpretation and assimilation of evaluation evidence. [101p.]




 *      *     *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues.  [DD/ KMC Area]
Washington DC USA

“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
PAHO/WHO Website
Equity List - Archives - Join/remove:

IMPORTANT: This transmission is for use by the intended
recipient and it may contain privileged, proprietary or
confidential information. If you are not the intended
recipient or a person responsible for delivering this
transmission to the intended recipient, you may not
disclose, copy or distribute this transmission or take
any action in reliance on it. If you received this transmission
in error, please dispose of and delete this transmission.

Thank you.

No comments: