The Magenta Book - Guidance for evaluation
Available online PDF [141p.] at: http://bit.ly/v9642L
“…….The Government is committed to improving central and local government efficiency and effectiveness, and in times of constrained public finances it is even more important to ensure that public funds are spent on activities that provide the greatest possible economic and social return. This requires that policy is based on reliable and robust evidence, and high quality evaluation is vital to this.
HM Treasury’s Green and Magenta Books together provide detailed guidelines, for policy makers and analysts, on how policies and projects should be assessed and reviewed. The two sets of guidance are complementary:
- the Green Book emphasising the economic principles that should be applied to both appraisal and evaluation, http://bit.ly/tHfSj4 and
- the Magenta Book providing in-depth guidance on how evaluation should be designed and undertaken.
The risk of not evaluating, or of poor evaluation, is that policy makers are not aware if policies are ineffective or, worse still, result in overall perverse, adverse or costly outcomes. If there is no good evaluation evidence to demonstrate it, then we cannot be confident that taxpayers’ money is being properly spent, even where policies are in reality highly effective. The knowledge we gain from good evaluation can be used to increase policy effectiveness and is essential in informing the development of new policies to achieve the best results…..”
Chapter 1 Key issues in policy evaluation
What is evaluation and what benefits can it bring?
What factors affect how a policy should be evaluated?
How evaluation fits into the policy cycle
Chapter 2 Identifying the right evaluation for the policy
How was the policy delivered? Process evaluation
What difference did the policy make? Impact evaluation
Did the benefits justify the costs? Economic evaluation
Why did what happened occur?
What type of evaluation for the policy?
How do evaluation questions relate to the underlying “logic” of the intervention?
Factors affecting the choice of evaluation approach
Chapter 3 Building impact evaluation into policy design
Thinking about impact evaluation when designing the policy
The role of comparison groups in identifying the impact of a policy
Chapter 4 What practical issues need to be taken into account when designing an evaluation
The main steps in the evaluation process
How to ensure an evaluation meets the requirements: governance and quality control
Timing of the evaluation
What types of resources are likely to be needed?
What level of resource should be dedicated to the evaluation
Chapter 5 The stages of an evaluation
The steps involved in planning and undertaking an evaluation
Chapter 6 Setting out the evaluation framework
Reviewing the existing evidence
Rapid evidence assessment
Meta-evaluation and meta-analysis
Making sense of existing and new evidence: simulation modelling
Chapter 7 Data collection
What is monitoring data and how can it contribute to evaluation?
New data collection
Designing data collection tools
Ethical and data protection considerations
Chapter 8 Process evaluation, action research and case studies
Evaluation to understand the implementation and delivery of policy
Why undertake a process evaluation, action research or case study?
Research methods to support process evaluation, action research and case studies
Choosing research methods
Chapter 9 Empirical impact evaluation
Introducing empirical impact evaluation
When are empirical approaches possible?
Designing policies for effective evaluation
Power of design
Strategies for analysing quasi experimental data
Thinking critically about the textbook techniques
Chapter 10 Drawing together and reporting evaluation evidence
How evaluation evidence may be used
Drawing together the evaluation evidence
Setting the evaluation results in a broader context
Future decisions and roll-out; scaling-up
Implications for evaluation planning
Reporting and disseminating findings
You could also see:
The book divided into two parts.
Part A is designed for policy makers. It sets out what evaluation is, and what the benefits of good evaluation are. It explains in simple terms the requirements for good evaluation, and some straightforward steps that policy makers can take to make a good evaluation of their intervention more feasible. http://bit.ly/sukpsl [28p]
Part B is more technical, and is aimed at analysts and interested policy makers. It discusses in more detail the key steps to follow when planning and undertaking an evaluation and how to answer evaluation research questions using different evaluation research designs. It also discusses approaches to the interpretation and assimilation of evaluation evidence. http://bit.ly/ryU7oH [101p.]
* * *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues. [DD/ KMC Area]
“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
Equity List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html
IMPORTANT: This transmission is for use by the intended
recipient and it may contain privileged, proprietary or
confidential information. If you are not the intended
recipient or a person responsible for delivering this
transmission to the intended recipient, you may not
disclose, copy or distribute this transmission or take
any action in reliance on it. If you received this transmission
in error, please dispose of and delete this transmission.