MAKING AN IMPACT
A Preferred Framework and Indicators to Measure Returns on Investment in Health Research
Report of the Panel on the Return on Investments in Health Research
Canadian
January 2009
Available online PDF file [136p.] at: http://www.cahs-acss.ca/e/pdfs/ROI_FullReport.pdf
"……Twenty three different organizations sponsored this assessment. They all share an interest in defining the impacts of health research and learning how to improve the returns on investments in health research. Our remit from these sponsors was: Is there a "best way" (best method) to evaluate the impacts of health research in
Based on our assessment, we propose a new impacts framework and a preferred menu of indicators and metrics that can be used for evaluating the returns on investment in health research. The CAHS impact framework demonstrates how research activity informs decision making, eventually resulting in changes in health and economic and social prosperity (left to right arrow). The framework also shows how research impacts feed back upstream, potentially influencing the diffusion and impacts of other research, and creating inputs for future research (right to left arrow).
This framework builds on the combined logic model and impacts approach of the "payback model" (Buxton, M.J., and Hanney, S.R., 1996 � adapted by CIHR in Canada in 2005 and 2008), revised by our panel into a "systems approach" to capture impacts (this is shown at the bottom of the Figure). It is designed to be used as a roadmap to track health�research impacts in five main categories:
1) advancing knowledge,
2) building capacity,
3) informing decision�making,
4) health impacts, and
5) broad socio�economic impacts………."
Content
Volume 1
The Chair's Perspective
Rationale (and CAHS Prospectus) for this Assessment
Executive Summary
Panel Recommendations
1. Chapter 1: Background
1.1. Defining Health Research
1.1.a. The Canadian Health Research Landscape
1.1.b. What Returns are Expected from Canadian Health Research?
1.1.c. Many Stakeholders, Many Views
1.1.d. Different Evaluation Purposes
1.1.e. Why Measure Returns on Investment for Health Research in
1.1.f. What Evaluation of "Returns" is
1.2. Are "Health Research Impacts" Already Defined Elsewhere?
1.3. Economic Evaluations to Date
1.4. Summary of the Landscape
2. Chapter 2: Frameworks
2.1. Rationale for a Framework to Understand Health Research
2.1.a. Definitions
2.1.b. Stakeholders Needs
2.2. A Review of Frameworks and Their Use
2.3. Developing a Health Research Evaluation Framework for
2.3.a. Identifying What Should be Modelled
2.3.b. Building a Framework for R&D Uptake
2.3.c. Impact Categories
2.3.d. Impact Categories and the Logic Model
3. Chapter 3: Strategies for Using the Framework
3.1. Using the Framework Appropriately
3.1.a. Avoiding Misuse of the Framework
3.1.b. The Four Pillars and the Framework
3.2. Costs of Evaluation
3.3. Issues in Evaluation: Attribution, the Counterfactual, Time�lags and Levels of Aggregation
3.4. Evaluation Methods
3.5. Data Collection
4. Chapter 4: Choosing Sets of Indicators and Metrics
4.1. Overview of Indicators and Metrics and Their Use
4.2. Defining "Appropriate Indicators"
4.3. Identifying Appropriate Indicators
4.3.a. Advancing Knowledge
4.3.b.
4.3.c. Informing Decision Making
4.3.d. Health Impacts
4.3.e. Broad Economic and Social Impacts
4.3.f. Theoretical examples of indicator sets for evaluation
5. Chapter 5: Conclusions
6. References
Volume 2
Appendices
The appendices present commissioned papers in areas where the report is not able to provide details, cover the background for the main report, and present the approach taken to the assessment process. The commissioned papers cover assessing the impacts of research in pillars II, III and IV but do not cover pillar I, since basic biomedical research is the area where most has been said on understanding the impacts of health research.
Appendix A: Commissioned Papers
- Pillar II: Clinical Research � "How to Optimally Measure the Impact of Health Research Funding in Clinical Research" by Ralph M. Meyer
- Pillar III: Health Services Research � "Estimating the Return on Investment from Health Services Research: A Theoretical and Empirical Analysis
Steven Lewis, Patricia J. Martens and Louis Barre
- Pillar IV: Population and Public Health Research �"Assessing the Return on
Methods and Metrics" by Alan Shiell and Erica Di Ruggiero
- Meso�Level Metrics for Impact � "Metrics for the Treatment Sector or Meso Level of the Canadian Health Care System" by Jerald Hage
- Ethics and Evaluating Health Research � "The Return on Investments (ROI) in Health Research: Ethical Aspects" by Michael McDonald and Bartha Knoppers
- Public Perspective on Health Research Funding � "Translating Science into Hope: The Public Perspective on Health Research Funding" by André Picard
- Health Research Evaluation Frameworks: An International Comparison by Philipp� Bastian Brutscher, Steven Wooding and Jonathan Grant
Appendix B: The Canadian Landscape for health research
Appendix C: Evaluation frameworks and methods
Appendix D: Issues for research evaluation
Appendix E: Indicators
Appendix F: Glossary
Appendix G: Methods
Appendix H: External Interviewees
Appendix I: Prospectus for a Major Assessment � The Return on Investments in Health Research: Defining the Best Metrics
|
|
Download the Appendices [pdf] 5.27 MB, (340 pages) ISBN: 978-0-9811589-1-4
Download the Assessment Summary [pdf] 1.15 MB, 24 pages
Pour Télécharger de « Résumé de l'évaluation » [pdf] 1.56 MB, 30 pages
Menu of Preferred Indicators [pdf] 1.56 MB, 8 pages
Website: http://www.cahs-acss.ca/e/assessments/completedprojects.php
* * *
This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues. [DD/ KMC Area]
"Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members".
------------------------------------------------------------------------------------
PAHO/WHO Website
Equity List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html
