P o lic y b rie f P o lic y b ri e f
Introduction
The Benchmarks of Fairness framework was conceived in the United States of America at the beginning of the 1990s to evaluate planned health insurance reforms. It is now used to evaluate the fairness of health sector reforms (Daniels et al. 2000, 2005) and several developing countries have already used it to strengthen their capacity to assess health-care reforms. In the context of social justice, the concept of fairness has three dimensions: equity in terms of access and financing, accountability, and clinical and administrative efficiency.
Nine benchmarks have been elaborated for the evaluation of fairness, each covering a dif- ferent aspect of health system performance and design. Among these, five relate specifi- cally to the issue of equity, two relate to clinical and administrative efficiency, and another two relate to administrative accountability and autonomy (Table 1).
Table 1. Benchmarks and dimensions used in the framework.
Benchmark Dimension of fairness measured
1. Intersectoral public health
Equity 2. Financial barriers to equitable access
3. Nonfinancial barriers to access
4. Comprehensiveness of benefits and tiering 5. Equitable financing
6. Efficacy, efficiency and quality of health care Efficiency 7. Administrative efficiency
8. Democratic accountability and empowerment Accountability 9. Patient and provider autonomy
Studies that use the Benchmarks of Fairness framework develop a locally-agreed-upon scoring method to rank the fairness of data on locally selected indicators of each of the nine benchmarks. The scores derived from assessments with these benchmarks are intended to reveal the complex effects of reforms on fairness. Baseline scores of the nine benchmarks can be compared with later assessments to draw inferences about the fairness of changes introduced by health sector reform.
This brief reports on how a team of researchers in the Philippines used the Benchmarks of Fairness framework to assess the fairness of sexual and reproductive health services in the province of Surigao del Sur, Mindanao. Indicators were primarily drawn from the World Bank’s second Women’s Health and Safe Motherhood Project but also from the Department of Health’s national health reform programme and the national health insurance scheme, PhilHealth. The study team worked in a highly participatory manner to systematically score the fairness of each indicator, using a number of technical working groups. A report and a manual were produced to guide future applications of the method in the Philippines (Social Sciences and Philosophy Foundation, 2008a, 2008b). This brief presents just a selection of the key findings.
SOCIAL SCIENCES &
PHILOSPHY RESEARCH FOUNDATION
UNIVERSITY OF THE PHILIPPINES, DILIMAN
Benchmarking the fairness of health sector reform in the Philippines
Benchmarking the fairness of health sector reform in the Philippines
© Susan Bender/Photoshare
Project site
The province of Surigao del Sur is located in the Caraga region of the Philippines (Mindanao) and is composed of 17 municipalities and two cities. The provincial population is estimated to be 409 468, representing 17.9% of the total population of Caraga region and 0.46% of that of the entire country (National Statistical Coordination Board, 1997–2008). The province was the 16th poorest Philippine province in 2006 (National Statistical Coordination Board, 2006a).
The National Statistical Coordination Board (2006b) indicated that 45.4% of families in the province lived below the poverty threshold.
The total fertility rate in Caraga province is 3.7, a little higher than the national average of 3.2 (National Statistics Office, 2007). The median age at birth of the first child among women aged 25–49 years was 22.5 years. Among women aged 15–19 years, 4.9% had begun childbearing. The percentage of married women with an un- met need for family planning was 29.7%.
Methods and activities
Implementation of the study was structured around five sequen- tial activities, each of which built on the conclusions of the former.
Technical working groups were convened for each activity. The first activity was the selection of benchmarks; the second was estab- lishment of criteria for selecting indicators; the third was speci- fication of indicators and data sources; the fourth was collecting existing data and organizing the findings; and the fifth was the scor- ing of fairness.
Each of the technical working groups included a broad range of stakeholders in health sector reform in Surigao del Sur, in the Caraga region and in the national programme. Officials from many branches of Government participated in addition to the Department of Health, such as the Department of Interior and Local Governments, and the National Commission on Indigenous Peoples. Representatives of civil society organizations and the academic community in Ma- nila were also involved in technical group deliberations. The project team functioned as a secretariat, organizing meetings for the advi- sory groups and preparing materials for their consideration based on recommendations. It was also responsible for drafting the final report and producing a manual.
Data sources
The use of existing indicators and data sets based the study within the health information system of the Department of Health and re- lated agencies. While this strategy was helpful in that it obviated the collection of new data, it made the study vulnerable to the short- comings of routinely available data. Table 2 shows that data were not available for many indicators. In addition, many of the data were not available in disaggregated form, so that meaningful compari- sons could not be made.
Table 2. Availability of data by indicator for each benchmark of fairness.
Dimension of fairness
Indicator Sub-indicator
No. % for which
data available No.
% for which data available
Equity 15 87 41 76
Efficiency 7 43 41 29
Accountability 7 71 18 33
Total 29 67 100 46
Scoring fairness: rules and procedures
With input from the technical working group, the project team de- signed a set of rules and procedures for rating the fairness of the assembled data. Each group used the same procedures to assess fairness on the basis of the data for each indicator, using the rules outlined below.
• A numerical score ranging from one to five was used to repre- sent the assessment of fairness.
• The score for fairness was assigned on the basis of:
- a comparison of data for Surigao del Sur with national data;
- a comparison of data for Surigao del Sur with regional data;
- improvement or deterioration over time;
- observations that certain subgroups were more or less likely to benefit or have better health outcomes (equity);
- the quality of the data and of measurements; and
- local results in relation to national goals or targets and the adequacy of the target for measuring fairness.
The numerical values for fairness were distributed as follows: a score of five implied the best possible achievable degree of fair- ness, as suggested by attainment of national goals or targets, uni- form benefits or health outcomes by different subgroups; and a score of one implied extreme bias or heavily skewed evidence for certain subgroups.
The evidence that was scored was derived from the best available data of the Department of Health and other relevant Government statistical sources on 29 indicators and 100 subindicators. For each subindicator, the study team prepared tables, graphs, figures or text descriptions of the evidence, using disaggregated data when pos- sible. The type of data varied widely. For each indicator, the national goal or target or both were also identified (when they existed) to serve as a reference for scoring fairness. Scoring was undertaken by subindicator, with a summary score for the indicator. The study team analysed the scores by various techniques to provide an over- all assessment of the fairness achieved for the benchmark.
Findings
This brief presents a few of the findings of the study. The full re- port is available from the Social Science and Philosophy Research Foundation (2008a).
Dimension 1: Equity
Figure 1 presents the fairness scores for a set of service delivery indicators on one of the benchmarks relating to equity. The fairness of four out of ten services was found to be borderline, and even those four did not achieve the midpoint of fairness; the others were judged to be quite unfair. In general, there is room for improvement, with a large proportion of services outside the range scored as fair.
Programmes that work on violence against women and children provided the least fair service, with a rating of one, indicating that this need has yet to be addressed adequately by the health serv- ices. For example, documentation of cases of domestic violence among the relevant agencies, such as the Department of Health, local government units, the police and the justice system, was not harmonized, resulting in a lack of reliable data on occurrence and prevalence.
The Benchmarks of Fairness study investigated how the PhilHealth insurance programme is working to reduce financial barriers in Surigao del Sur. Overall, there is a large difference between the proportion of the population who live below the poverty line (45.4%) and the proportion of the population who are covered by PhilHealth (19.6%): More needs to be done to achieve better levels of financial equity in using health services in this province.
Figure 2 shows the extreme variations in allotments of the Phil- Health Indigency Fund in different municipalities in Surigao del Sur, which led the raters to assess the fairness of measures to reduce financial barriers as only moderately successful. While there was an increase in funds in several sites (e.g. San Miguel, Barobo, Cantilan, Hinatuan and Cortes), the amount allocated to health insurance for indigent persons remained constant in other areas (e.g. Tandag City and Marihatag). The annual contributions of most municipalities were inconsistent during the five-year period. Even the Provincial Health Office of Surigao del Sur provided funds only in 2002 and 2004. Twelve of the 19 municipalities had no data on their allot- ment to the PhilHealth Indigency Fund. Although this might repre- sent poor reporting, the Technical Working Group found it equally likely that these municipalities did not make an allotment to the programme and hence had nothing to report.
Dimension 2: Efficiency
The range of scores for the different indicators of one of the bench- marks of efficiency (i.e. the degree to which reform has improved the quality of health service delivery) is representative of that for the other benchmarks. The fairness of reform activities was moder- ate to strong, ranging from two to four (Figure 3). The raters gave the lowest scores to the referral mechanisms, support systems for the World Bank Women’s Health and Safe Motherhood Project and the emergency evacuation system. They agreed that the score of 2 was high, as these systems are still being set up or are not yet fully operational. The indicators with higher scores were those for ac- creditation of health providers after regular assessment of quality of
Figure 1. Ratings for benchmark 1, indicator 1: Degree to which reform has advanced the health status of the population, especially women, adolescents and children.
0 1 2 3 4 5 Fertility rate
Immunization Violence against
women and children
Contraceptive use
Nutrition
Selected causes of morbidity
Crude birth rate Infant mortality
rate Perinatal
deaths Unmet FP need
Figure 2. Trends in allotment of funds for the PhilHealth Indigency Fund by selected municipalities and the Surigao del Sur Provincial Health Office, 2000–2004.
Tandag San
Miguel Barobo
Cantila n
Hinatuan Marihatag
Cortes Suriga
o del Sur
(PHO) MUNICIPALITY
100,000 200,000 300,000 400,000
AMOUNT IN Php
2000 2001 2002 2003 2004
Dimension 3: Accountability and empowerment
Although there were many indicators in government programme documents that related to this dimension, upon investigation the study team found very little data that corresponded to the indica- tors. Sufficient data were found to permit an assessment of how fair the health system was with respect to only one benchmark (‘client and provider autonomy’). Data were available for only two of the indicators for this benchmark and each showed incomplete reporting, with gaps and missing values (Figure 4). This is in itself an important finding: more attention is needed to collecting appro- priate data in a consistent manner to ensure adequate monitoring of accountability and empowerment.
However, the results show that the rating was fairly strong, with scores between 3.5 and 4. These scores are due, however, prima- rily to the existence of standards and guidelines for the services that must be available in health-care facilities and not to use of the guidelines in monitoring or regulating the health system at local level.
Discussion
How relevant is the Benchmarks of Fairness framework to the Philippines?
The advisers and stakeholders in the study agreed that the analyt- ical framework and scoring procedures used in the Benchmarks of Fairness study are relevant to the Philippines. Although the availa- bility and accessibility of data were severely limited, it was agreed that the study had been useful for stimulating discussion about fairness and social justice in the health sector, supporting the emphasis in the Department of Health’s health sector reform pro- gramme on equity and efficiency. Scoring helped to contextualize
‘fairness’ for both Government and non-Government representa- tives. The value of conducting a study using the Benchmarks of Fairness framework in the Philippines thus lay more in its conduct than in the comprehensiveness of its results.
The conceptualization of equity captured by the term ‘fairness’
led to a discussion of other ways of assessing equity, beyond the wealth quintiles most commonly used in the Philippines. It suggested that an appreciation of fairness can improve planning, monitoring and evaluation in the health sector. Participants in the working groups and members of the steering committee who lis- tened to these discussions will be able to act on the lessons learnt from the study.
Figure 4. Distribution of ratings for indicator 9.1: degree to which reform has increased client autonomy.
Rating Providers that
promote client autonomy
Wide array of client choices
5 4 3 2 1 0
Sub-indicator
Figure 3. Distribution of ratings for indicator 6.1: Degree to which reform has improved the quality of health service delivery.
Assessment of quality of service
Mechanisms for regular assessment of quality
of services
Operational emergency evacuation system
Support systems for the World Bank Women’s Health and Safe Motherhood
Project
Environmental and health care waste management measures
Operational referral mechanisms
Accreditation of hospitals and facilities
Accreditation of health providers Training of health providers
0 1 2 3 4 5
service, reflecting the operational efficiency of the PhilHealth pro- gramme. These findings suggest that the health system is achieving a moderately high degree of fairness with respect to the delivery of sexual and reproductive health services and will score higher once the referral and evacuation systems become operational.
What results were most useful for policy and programmes?
Availability of data
The project showed that extensive data are available on the health system and on the population’s health status, not just from the De- partment of Health but also from agencies such as PhilHealth and the Food Nutrition Research Institute. However, although there is a wealth of data there is a poverty of information. One problem was the scarcity of data disaggregated by sex and by sub-provincial or other groupings. Furthermore, decentralization of the Department of Health made it difficult to link and integrate data from different levels of the health system. In many settings, the study team found disorganized statistics at municipal level, most of which were not transmitted to provincial, regional or national levels.
Equity
Although several of the indicators were assessed as moderate to very fair, particularly in reducing financial barriers and promoting health equity, the assessment showed uneven performance of dif- ferent health programmes, resulting in unfair coverage and access to sexual and reproductive health services.
Efficiency
The technical working groups concluded that improving the health system’s efficiency would have a positive effect on the population’s health. The lowest ratings were given to the mechanisms targeted by the health sector reform programme for development, such as referral mechanisms and the emergency evacuation system. The limited availability of data on these indicators is surprising, given the reform programme’s emphasis on equity and efficiency. More needs to be done to collect the appropriate types of data and to have them available in easily accessible form by the Department of Health and sister agencies in government.
Accountability
The Department of Health is quite simply not collecting sufficient information on how the health sector is working to promote condi- tions of transparency and accountability. This is particularly trou- bling because the scant evidence that does exist indicates that progress is being made and that efforts are underway to create conditions of being accountable. With additional attention to this dimension of fairness, the Department of Health could very well be able to provide evidence of success.
Performance evaluation
Although this study was not intended as an evaluation, the scores in fact represent an assessment of progress achieved in meeting targets and goals. The Benchmarks of Fairness analytical frame- work was useful for stimulating discussions of equity, and a similar
‘fairness lens’ could be used in other evaluations. For example, both the Department of Health and other Government agencies now emphasize performance-based budgeting (allocating funds on the basis of performance), using data from accreditation schemes (e.g. Sentrong Sigla of the Department of Health and PhilHealth’s
‘Benchbook on Performance Improvement of Health Services’), lo- cal government units self-assessment tools (the Department of the Interior and Local Government performance management system) and scorecards (for ‘monitoring and evaluation for equity and ef- fectiveness’ and that of local government units). The scoring pro- cedures used in this Benchmarks of Fairness study could be used to assess these different performance measures, creating oppor- tunities for discussion of the same results from the viewpoint of fairness and social justice.
Bibliography
Daniels N et al. (2000) Benchmarks of fairness for health care re- form: a policy tool for developing countries. Bulletin of the World Health Organization, 78:740–750.
Daniels N et al. (2005) An evidence-based approach to benchmark- ing the fairness of health sector reform in developing countries.
Bulletin of the World Health Organization, 83:1–7.
National Statistical Coordination Board (1997–2008) Active Stats, PSGC Interactive, Province: Surigao del Sur. Retrieved August 27, 2008, from http://www.nscb.gov.ph/activestats/psgc/regview.
asp?region=16
National Statistical Coordination Board (2006a) 2006 Philip- pine Poverty Statistics – Annual Per Capita Poverty Thresholds, Poverty Incidence and Magnitude of Poor Families: 2000, 2003, 2006. Retrieved August 27, 2008, from http://www.nscb.gov.ph/
poverty/2006_05mar08/table _1.asp
National Statistical Coordination Board (2006b) 2006 Poverty Sta- tistics – Ranking of Provinces Based on Poverty Incidence Among Families: 2000, 2002, 2006. Retrieved August 27, 2008, from http://www.nscb.gov.ph/poverty/2006_05mar08/table _24.asp National Statistics Office (2007) 2006 Family Planning Survey. Ma- nila: National Statistics Office.
Social Sciences and Philosophy Foundation (2008a) Final report:
benchmarking the fairness of health sector reform in the Philippines.
Quezon City, University of the Philippines (bof_phils@yahoo.com).
Social Sciences and Philosophy Foundation (2008b) Manual on benchmarking the fairness of health sector reform in the Philippines.
Quezon City, University of the Philippines (bof_phils@yahoo.com).
Further reading
Impact of provider incentive payments on reproductive health services in Egypt. World Health Organization, 2009.
WHO/RHR/09.04.
Public policy and franchising reproductive health: current evidence and future directions. Guidance from a technical consultation meeting. World Health Organization, 2007. ISBN 978 92 4 159602 1.
Public–Private Partnerships: Managing contracting arrangements to strengthen the Reproductive and Child Health Programme in India.
Lessons and implications from 3 case studies.
World Health Organization, 2007. WHO/RHR/07.15.
Financing sexual and reproductive health-care services.
World Health Organization, 2006. Policy Brief 1.
The effect of maternal–newborn ill-health on households:
economic vulnerability and social implications. World Health Organization, 2006. ISBN 92 4 159448 6 /ISSN 1990-5130.
The costs of maternal–newborn illness and mortality. World Health Organization, 2006. ISBN 92 4 159449 3 /ISSN 1990-5130.
Impact on economic growth of investing in maternal–newborn health. World Health Organization, 2006.
ISBN 92 4 159449 9 /ISSN 1990-5130.
These publications can be found on the WHO website:
www.who.int/reproductivehealth
Acknowledgements
This brief was written by Dale Huntington, Michael L. Tan and Maria Theresa D. Ujano-Batangan, and draws from the final technical report “Benchmarking the Fairness of Local Health Sector Reform Initiatives in the Philippines”, available from the social Sciences and Philosophy Research Foundation, University of the Philippines, Diliman.
Institutional References:
Department of Reproductive Health and Research World Health Organization
Avenue Appia 20, CH-1211 Geneva 27 Switzerland
Social Sciences and Philosophy Research Foundation, inc.
2nd Floor, Benton Hall, Roxas Avenue University of the Philippines, Diliman 110 Quezon City
Philippines
Benchmarking the fairness of health sector reform in the Philippines
© World Health Organization, 2009
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.
For further information contact:
Department of Reproductive Health and Research World Health Organization
Avenue Appia 20, CH-1211 Geneva 27 Switzerland
Fax: +41 22 791 4171
E-mail: reproductivehealth@who.int www.who.int/reproductivehealth
