[EQ] Making Sense of Health Rankings

Making Sense of Health Rankings

 

Canadian Institute for Health Information, (Ottawa, Ont.: CIHI, 2008).

ISBN 978-1-55465-329-4 (PDF)

 

Available online PDF [29p.] at:

http://secure.cihi.ca/cihiweb/products/MakingSense_HealthRankings_web_secure_e.pdf

 

“….As Canadians, we all want to optimize our health status and be assured of health care that is effective, timely and delivered by compassionate providers attuned to our needs and those of our families. In an era of increasingly complex medical care and escalating costs, provincial and territorial governments are attempting to balance these public expectations with their pressing fiscal responsibilities. Health care decision-makers, including provincial and territorial ministries of health, local health authorities and hospital administrators, are increasingly relying on a broad range of indicators to gauge the health of the Canadian population, the quality of hospital care and the performance of health care systems.

 

Reports that rank the health of Canadians and Canada’s health care systems according to these indicators are widely reported in the media. These reports often attempt to condense a complicated array of statistics on population health and health care into a relatively simple number, a rank that is used to make

international and provincial comparisons. But in recent years, these reports have often been inconsistent. In one report, Canada is ranked 11th among 24 developed countries, and in another Canada is ranked 23rd out of the 30 countries compared on the basis of the performance of their health care systems.1, 2 Unlike a familiar economic indicator—the gross domestic product (GDP), which represents a complex entity (the total market value of all goods and services produced in a country in a given year) with a single number calculated according to an internationally agreed upon methodology—rankings of health and health care are not yet standardized nor well understood.

 

This methodology paper aims to improve readers’ understanding of ranking reports. It outlines the components and processes that underlie health rankings and explores why such rankings can be difficult to interpret. The paper concludes with a checklist to help…”

 

readers judge and compare the value of health-related rankings.

 

Step 1: Assess the soundness of the conceptual framework

Does the ranking scheme’s conceptual framework cover the areas of health and health care that are relevant to the purpose of the ranking?

Step 2: Assess the indicators chosen to measure selected aspects of health and health care

Are the indicators of health or health care used in the ranking consistent with the conceptual framework?

Are the measures used for the selected indicators meaningful and valid?

Step 3: Assess the data quality

Are data accurate, reliable, complete, comparable and free from bias?

Are data elements defined and collected so that “apples to apples” comparisons are being made?

Step 4: Examine soundness of methods

Are meaningful differences in performance distinguishable?

Are absolute and relative comparisons available for review?

Have appropriate adjustments been made for underlying differences in the populations being compared?

Is the way specific measures are combined in the ranking scheme clear?

Is the specific formula, along with any weights used to combine individual measures or indicators, based on clear and reasonable principles?

Are differences in performance statistically significant?

Have other statistical issues been appropriately handled (for example, adjustments for correlated measures or handling outlier values and ties)?

Have the authors of the report reduced the potential for bias through full disclosure of ranking methods and peer review?

 

Content:

Executive Summary
Introduction

Evaluating Health Rankings

Summary

References

 

 *      *      *     *

This message from the Pan American Health Organization, PAHO/WHO, is part of an effort to disseminate
information Related to: Equity; Health inequality; Socioeconomic inequality in health; Socioeconomic
health differentials; Gender; Violence; Poverty; Health Economics; Health Legislation; Ethnicity; Ethics;
Information Technology - Virtual libraries; Research & Science issues.  [DD/ KMS Area]

“Materials provided in this electronic list are provided "as is". Unless expressly stated otherwise, the findings
and interpretations included in the Materials are those of the authors and not necessarily of The Pan American
Health Organization PAHO/WHO or its country members”.
------------------------------------------------------------------------------------
PAHO/WHO Website
Equity List - Archives - Join/remove: http://listserv.paho.org/Archives/equidad.html

 

 

 

    IMPORTANT: This transmission is for use by the intended recipient and it may contain privileged, proprietary or confidential information. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you may not disclose, copy or distribute this transmission or take any action in reliance on it. If you received this transmission in error, please notify us immediately by email to infosec@paho.org, and please dispose of and delete this transmission. Thank you.