July 2019
The Transformation Agenda
6 series
Towards a Stronger Focus on Quality and Results:
The Story of Programmatic
Key Performance Indicators
Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO;
https://creativecommons.org/licenses/by-nc-sa/3.0/igo).
Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”.
Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization.
Suggested citation. The Transformation Agenda Series 6: Towards a Stronger Focus on Quality and Results: Programmatic key performance indicators.
Congo, Brazzaville: WHO Regional Office for Africa; 2019. Licence: CC BY-NC- SA 3.0 IGO.
Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris.
Sales, rights and licensing. To purchase WHO publications, see http://apps.who.
int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing.
Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.
General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use.
Designed and printed in Congo ISBN: 978-929031322-9
© WHO Regional Office for Africa 2019
CONTENTS
FOREWORD iii
1. BACKGROUND 1
2. KPIs AT WORK 3
3. USING KPIs TO MANAGE FOR RESULTS 15
4. FUTURE PERSPECTIVES 19
ii
The Transformation Agenda of the World Health Organization Secretariat in the African Region 2015–2020 strives to deliver “the WHO that the staff and stakeholders want”. Over the past five years, WHO in the African Region has made a concerted effort to transform itself into a forward-looking, proactive, responsive, results-driven, transparent, accountable, and appropriately resourced organization. Member States, development partners, donors, and other stakeholders have stood shoulder to shoulder with WHO in advancing this agenda – for this I am sincerely thankful.
Implementation of the Transformation Agenda began in 2015, coinciding with the global adoption of the Sustainable Development Goals. Goal 3 is to ensure healthy lives and promote well-being for all at all ages. The WHO Secretariat in the African Region used this opportunity to institutionalize the Transformation Agenda by developing The Africa Health Transformation Programme 2015–2020: a vision for universal health coverage.
This document serves as the strategic framework for guiding WHO’s contribution to the sustainable development platform in Africa. This framework aims to ensure universal access to a package of essential health services in all Member States of the Region and thus achieve universal health coverage with minimal financial, geographic and social obstacles to services.
FOREWORD
iv
To date, we can see the impact of the Transformation Agenda in three key areas:
(1) Health security has improved through increased regional and country capacity to promptly detect and effectively respond to public health threats. Many of the outbreaks we have faced have been rapidly contained.
(2) Member States are progressing towards universal health coverage through efforts to strengthen health systems. Improving access to cost–effective health interventions is leading to improved health outcomes in Member States.
(3) A value-based organizational culture is emerging where harassment is openly addressed, and a respectful working environment is in place. We are seeing
fundamental shifts in our ways of working, thinking and engaging with others as well as increased accountability, effectiveness and transparency, and tangible results in countries.
The Transformation Agenda Series seeks to share the key achievements of this initiative in six booklets:
•
The Transformation Agenda Series 1: Enhancing the country focus approach for greater health impact;•
The Transformation Agenda Series 2: Strengthening partnerships towards universal health coverage;•
The Transformation Agenda Series 3: Improved capacity to tackle epidemics in the WHO African Region – lessons from the 2016 yellow fever outbreaks in Angola and the Democratic Republic of the Congo;•
The Transformation Agenda Series 4: Sustained progress towards polio eradication in the WHO African Region;•
The Transformation Agenda Series 5: Promoting efficiency, accountability and value for money – the story of managerial key performance indicators; and•
The Transformation Agenda Series 6: Moving towards a stronger focus on quality and results – the story of programmatic key performance indicators.As we celebrate the gains made, I would like to express my gratitude to our stakeholders – Member States, development partners, donors, foundations and others – for walking the talk with us. Now, these gains need to be consolidated, sustained and stepped up in the coming year and beyond.
It is my expectation that the achievements highlighted in this series will spur us all to continuously strive towards ensuring healthy lives and promoting well-being for all people in the African Region and beyond by achieving universal health coverage, addressing health emergencies, and promoting healthier populations.
Dr Matshidiso Moeti Regional Director for Africa World Health Organization July 2019
vi
1 World Health Assembly decision on WHO Reform (WHA65(9);
URL:http://apps.who.int/gb/
ebwha/pdf_files/WHA65-REC1/
A65_REC1-en.pdf#page=25 2 The Transformation Agenda
Series 5: Promoting Efficiency, Accountability and value for Money – the Story of the Managerial Key Performance Indicators
1. BACKGROUND
Globally, there has been a significant shift towards demonstrating tangible results of development actions. WHO also faces intensified calls for accountability to citizens for how resources are used, results achieved, and how effective these results are in bringing about progress in health outcomes. This was given due attention when the global WHO reforms1 was initiated in 2012.
As part of the Transformation Agenda, the WHO Regional Office developed the African Region Results Framework in 2015 to measure WHO’s contribution to Africa’s health.
The framework is intended to guide the Secretariat to better serve Member States while improving transparency, reinforcing accountability, and demonstrating results on a journey of continuous improvement.
To become more results-driven and accountable, the WHO Regional Office for Africa developed and introduced managerial key performance indicators (KPIs) in mid-2015, within the framework of the Accountability and Internal Control Strengthening (AICS) project2 to measure performance and to assess compliance of operations with WHO rules. The four broad objectives of the AICS project are to: (1) strengthen the adequacy and effectiveness of internal controls; (2) improve accountability, transparency and compliance; (3) enhance the performance of individual staff and budget centres; and (4) measure, monitor and report on progress. Initially, 12 managerial KPIs for country offices were introduced in 2015. These related to: finance, budgeting, security; administrative services, human resources management; and audit and compliance.
2
In 2016, the number of KPIs was expanded to 23 and the Region began linking the performance of budget centres, budget centre managers and staff working in the enabling functions to KPI achievement using the WHO performance management and development system (PMDS). Assessment of the implementation of managerial KPIs showed improved administrative effectiveness and efficiency of WHO country offices.
In 2017, the Regional Director decided to expand the results framework with additional KPIs for technical programmes and enabling functions for regional and country office budget centres to better measure and articulate the contributions of the WHO Secretariat in the Region to Africa’s health development so as to better serve Member States. The Bill & Melinda Gates Foundation has supported the development of KPIs in the Region.
For a long time, WHO has faced challenges in measuring the outputs and outcomes of our support to Member States. The KPIs offer all of us an opportunity not only to measure results but also to focus our work on what will make a difference in our Region
Dr Matshidiso Moeti, WHO Regional Director for Africa
“ ”
2. THE PROGRAMMATIC KEY
PERFORMANCE INDICATORS AT WORK
The programmatic KPIs were determined through a participatory process starting with a one-day seminar of the Region’s senior management in May 2016. This was followed by a series of consultations with two working groups:
one for WHO representatives and another for staff from the Regional Office. The process was structured in three main phases (Figure 1). In Phase 1, a long list of KPIs (more than 200) was developed covering the work of the technical clusters and enabling functions. This long list was shortened to around 50 KPIs during Phase 2. By May 2017, a group of technical experts further refined the list to 43 KPIs.
Figure 1. Definition of programmatic KPIs – timelines
4
To ensure alignment of KPIs with national, regional and global priorities, the following selection criteria were established based on feedback from WHO representatives and Regional Office cluster directors:
• Aligned with the SDGs and WHO regional strategic objectives
• Aligned with the WHO programme budget and workplans
• Aligned with priorities of Member States and partners as articulated in country cooperation strategies and allocated programme funding
• Demonstrated significant WHO contribution with tangible results
• Achievable – availability of adequate human and financial resources to deliver results
• Easy to monitor with reliable data sources; data quality and controls in place
• Adapted to WHO in the African Region as an Organization and aligned to WHO’s mandate
• Makes sense to the WHO workforce and encourages personnel motivation.
Figure 2. An example of the WHO African Region results chain
HIV/AIDS
WHO AFR’s contribution Key Performance Indicator Thresholds @ Country level Support countries
in adaptation and implementation of WHO HIV Treatment and care guidelines 2015
@ Regional Level: Number of countries implementing WHO 2015 consolidated guidelines for HIV/AIDS treatment
@ Country Level: 2015 WHO HIV treatment guidelines implementation status
Green:Policy adopted, guidelines adapted and endorsed, "TREAT ALL"
implementation in progress Yellow:Policy adopted, guidelines adapted and endorsed, but no "TREAT ALL" implementation
Red:Policy and/or Guidelines not endorsed Baseline – EoY 2014 Target– EoY 2019
Legend Not applicable
6
The indicators consist of 11 “enabling” KPIs and 32 programme-related KPIs (Table 1).
The 43 prioritized KPIs span WHO clusters and programmes.
Table 1. Programmatic KPIs across WHO clusters, programmes and enabling functions
Cluster/programme Number of programmatic KPIs
Communicable Diseases (CDS) 6
Noncommunicable Diseases (NCD) 4
Family and Reproductive Health (FRH) 5
Health Systems and Services (HSS) 5
WHO Health Emergencies programme (WHE) 6
Health Promotion and Determinants (HPD) 2
Polio (PEP) 4
WHO enabling functions / corporate services Number of programmatic KPIs
External Relations, Partnerships and Governing Bodies (EPG) 3
Planning, Budgeting, Monitoring and Evaluation (PBM) 3
Communications (COM) 1
Human Resources Management (HRM) 2
Audit and Compliance (COT) 2
A reference sheet was developed for each indicator highlighting its measurement, data sources, and frequency of reporting. A handbook – WHO African Region Results Framework;
Measuring the Organization’s Contribution to Africa’s Health; Pocket Guide3 – was also published in 2017.
3 WHO African Region Results Framework; Measuring the Organization’s Contribution to Africa’s Health; Pocket Guide;
World Health Organization 2017;
ISBN 978-929023358-9
4 By 2020: 90% of all people living with HIV will know their HIV status;
90% of all people with diagnosed HIV infection will receive sustained
The handbook summarizes the KPI reference documents and provides information on the WHO African Region’s contribution to each KPI, specific KPIs at regional and country levels, thresholds, baselines (end of 2014), and targets (end of 2019).
All countries in the Region are expected to monitor the following 12 priority areas:
1. HIV treatment (90–90–90 targets)4
2. Noncommunicable disease prevention and control plan
3. Reproductive, maternal, newborn, child and adolescent health (RMNCAH) plan development
4. Coverage with three doses of diphtheria-tetanus-pertussis-containing vaccine (DTP-3)
5. Health workforce coverage
6. Integrated Disease Surveillance and Response (IDSR) 7. International Health Regulations (IHR)
8. African Regional Certification Commission (ARCC) for polio eradication 9. Percentage of resource mobilization out of allocated budget
10. Utilization target achievement
11. Communication strategy and outputs development 12. Managerial KPIs (overall score).
In addition, each country office selects seven priority areas to address country-specific priorities (examples in Table 2). The seven country-specific KPIs are selected based on the following criteria:
• strategic (i.e. a top priority from a government/population perspective);
• aligned with the WHO programme budget/WHO country cooperation strategy;
8
Table 2. Examples of country-specific priorities for KPIs Democratic Republic of the Congo Sierra Leone
CDS – Tuberculosis WHE – Health emergencies
CDS – Malaria FRH – Quality of care
CDS – Neglected tropical diseases CDS – Malaria FRH – Integrated community case
management
CDS – Tuberculosis
HSS – National health accounts HSS – Service availability and readiness assessment (SARA)
Polio – Polio surveillance HSS – National health accounts WHE – Incident management system GMC – Gender equity
Figure 3 summarizes the key steps in institutionalizing the KPIs, from defining the scope of programmatic KPIs, to obtaining support and buy-in, to driving commitment and establishing the required infrastructure. Capacity-building was a key component at each step.
• Identify stakeholders, costs and benefits of KPIs
• Define KPI Results Framework for Africa Region Scope and define programatic KPIs
• Engage leadership, establish KPI governance structure including KPI sponsors, comminicate and advocate the need for change
• Regional Office awareness sessions Achieve support and buy-in
• Engage WHO- AFR staff in planning the vision for change, validate costs and benefits
• KPIs selection at country level
• KPI roll out in countries
• WHO Country Office KPI Governace Structure
• Linking KPIs to staff PMDS Drive commitment
• Introduce new tools, technology, reward systems, trainings
• KPI quarterly progress reporting
• KPI monitoring tool
• Region-wide KPI capacity building workshops Establish KPI Infrastructure
• Measure progress, domonstrate value, communicate success, take corrective action
• Online Tool
• KPI Annual Performance Reports
• Integration with other regional reporting instruments Sustain
programmatic KPIs Figure 3. Institutionalizing KPIs in the WHO African Region – the five-step approach
10
Following the definition of the KPIs, the Regional Office organized 15 country office visits.
These visits aimed to gain buy-in from country office teams and start the process of institutionalizing the KPIs. The team facilitated the creation of governance structures for managing and monitoring progress against the KPIs and introduced reporting using an Excel template.
A KPI online tool was then introduced and three workshops conducted in March, April and May 2018 for KPI coordinators from all 47 country offices in the Region. The workshops introduced the new tool, provided guidance on reporting for results, and obtained feedback on the Excel-based reporting process. Feedback from the workshops shaped subsequent enhancements of the online tool, including incorporating a directory of focal points in the Regional Office and country offices to promote closer collaboration.
Commitment and ownership by leaders and managers was a key driving force. Regional KPI sponsors were identified at senior management level to achieve support and buy-in at all levels. These are usually technical cluster directors and heads of units and programmes.
A KPI advocacy strategy and communication tools were used to dispel misconceptions, mainstream results-focused approaches and promote the Region’s vision for change. By further automating and customizing it to suit staff needs, the reporting tool has enhanced the utilization of the KPIs.
To support institutionalization of the use of KPI data to drive results-based health management in the Region, WHO in August 2018 launched the Tool for African Region Results (TAR), an online platform that is used to collect and report on KPI performance data on a quarterly basis (Figure 4). The TAR allows for automated KPI reporting and enhances the utilization of KPI performance data for decision-making. The tool displays easy-to-access data for tracking performance trends in real time. This information is used by staff in making decisions, such as correcting course to address unsatisfactory performance.
Figure 4. Screenshot of the WHO Tool for African Region Results (TAR)
12
All 47 country offices report KPI results in the TAR on a quarterly basis. KPI quarterly reporting provides the basis for assessing whether expected results are on track to be achieved. These reports also facilitate objective assessment of KPI targets. KPIs are incorporated in each staff member’s annual objectives, using the WHO electronic performance management and development system (ePMDS). This facilitates clear lines of accountability and direct linking of each person’s work with organizational objectives.
Figure 5 describes the roles and functions in the KPI reporting process. Figure 6 shows the validation workflow for data generated in the TAR. The workflow is designed to improve data quality and to reduce the probability of disputes emanating from data inaccuracy.
The TAR provides quarterly trend reports for each KPI as well as maps depicting the regional evolution for each KPI. The trend report compares the KPI baseline (fourth quarter 2014 for most KPIs) and KPI achievement at annual and quarterly intervals.
The trend report indicates, with arrows and colour-coding, whether there has been an improvement, for example from red (off-track) to green (on-track), no change (same colour), or regression from green to yellow (at risk). Point-in-time maps of progress against KPIs are a useful reference in telling the story of how work has progressed within and across countries. This information is useful when WHO country office teams discuss strategic and operational activities with ministries of health, donors and partners as they provide evidence of the country’s performance against national targets.
Efforts are underway to improve the functionality of the TAR, with enhancements expected by the third quarter of 2019. These will include a new set of reports categorized into management (targeting programme managers who want to track the programme budget with focus on results, resources and risks), user-based (for various audiences and to reinforce the ability to assess trends and communicate WHO’s contribution) and administrative reports (to aid compliance and monitor reporting and utilization of the tool).
Table 3. Roles and functions in the KPI validation workflow Role in the KPI
process Function Number and location
Country KPI Focal Point Reports on assigned KPIs every quarter to
the KPI Coordinator 20 per office across 47 country offices (one focal point per KPI)
Country KPI Coordinator First-level reviewer of reports, requests
adjustments as necessary One per country office WHO Representative Second-level reviewer, assesses whether
country achievements are accurately reported, requests adjustments as
necessary and then clears for the reports to proceed to the Regional Office
One per country office
Regional KPI Focal Point Checks the submitted reports for
consistency, sends them to the KPI Sponsor One in the Regional Office KPI Sponsor Reviews reports: when the KPI Sponsor
approves a report it is published on the TAR 20 in the Regional Office (one per KPI)
14
KPI Focal Point
Saved Send for clearance KPI Coordinator
WHO Representative Validated
Request review
Approved
Regional Office Focal Point
Request review Approved
KPI Sponsor
Request review
Validated Data Published
Request review
Figure 5. KPI validation workflow (from country offices to the
Regional Office)
3. USING THE KPIs TO MANAGE FOR RESULTS
KPI monitoring has become an integral part of decision-making across the Region. A culture of using evidence to showcase achievements is developing in WHO country offices.
For example, progress on KPIs informs: strategic decisions in monthly management meetings; development and monitoring of strategic and operational plans; and resource mobilization and allocation. KPI monitoring has also opened up opportunities for optimizing processes and interventions, enhancing engagement with government counterparts and partners, and improving the quality of donor reports.
Routine KPI reporting has encouraged WHO to further institutionalize data gathering and performance monitoring. The capacity of WHO country offices has also been strengthened through improved integration and alignment of programmes with programmatic KPIs, national health development goals and the Sustainable Development Goals.
Staff members using KPIs on a regular basis have also noted benefits at individual and organizational levels. At the individual level, benefits include: refocused objectives in the ePMDS; clarity in pinpointing achievements for the year; improved understanding of WHO’s work in general due to the teamwork fostered through collaboration in finalizing KPI reports; and a heightened sense of achievement. At the organizational level, teams have noted increased transparency in decision-making and allocation and use of resources;
16
The use of KPIs is improving partnerships with line ministries, development partners, donors and other stakeholders. In discussions with ministries of health, KPIs enable closer alignment with government-defined priorities. KPIs provide a basis for donors and partners to assess WHO’s performance against prioritized health goals. For example, the Department for International Development of the United Kingdom (DFID) has included three African Region-specific indicators in their results framework for WHO. This framework is linked to 50% of their core voluntary contributions to WHO’s performance.
The KPIs have improved our way of working with the Ministry of Health and enhanced collaboration with the County Office”
– KPI Focal Point, WHO Country Office
“
”
Improving
transparency Reinforcing
accountability Demonstrating results
This new set of KPIs will help us to better serve our Member States while:
Engaging on a continous improvement journey
The use of KPIs has helped in identifying gaps in capacity, reporting compliance and data validation. In response, the African Region has developed tools, resource materials and tailored capacity-building workshops; designed TAR step-by-step user guides in English and French; introduced a help desk to respond to KPI-related queries; integrated a five- step data validation process from country to regional level; enhanced the TAR to promote a feedback loop between the Regional Office and country offices; and harmonized KPI reporting with other internal reports to minimize duplication and reporting fatigue.
Communication on the effectiveness of WHO’s contribution to health outcomes in the Region has significantly improved. The quality of quarterly KPI progress reports continues to improve and WHO is leveraging this to foster and strengthen policy dialogue and strategic partnerships for improved health outcomes across the Region.
The results framework and KPIs provide information on progress in programme implementation, evidence of achievements, and recognition of staff contributions. The programmatic KPIs are enabling the WHO Secretariat in the African Region to transform itself into a results-driven Organization by emphasizing the Organization’s accountability and shared leadership for results. These tools are making the Secretariat more responsive and fit for purpose as they provide a strong evidence base to inform discussions on policy solutions and resource allocation; identify neglected programme areas and marginalized populations; highlight best practices and reorient actions where necessary; and ramp up advocacy and galvanize inter-agency cross-collaboration.
18
4. FUTURE PERSPECTIVES
Implementation of the WHO African Region Results Framework, including programmatic key performance indicators, has been a major step in ongoing efforts to foster pro- results values across the Region. However, a number of challenges persist. Information technology network connectivity issues have, in some instances, hampered smooth reporting. Country offices have complained of a “reporting overload” as KPI reporting has been added to the already existing statutory reporting, including semi-annual, mid-term and end of biennium reports and quarterly reports to the Regional Director. As the Twelfth General Programme of Work, 2014–2019 draws to a close and the Thirteenth General Programme of Work, 2019–2023 (GPW 13) commences, KPIs will need to be realigned to clearly articulate the Region’s contribution to the Organization-wide “triple billion” goal.
The alignment process has already begun with the convening of a consultative meeting engaging the 47 country office KPI coordinators and the Regional Office KPI focal points in February 2019. The objective of the meeting was to review the current KPIs for relevance and propose new indicators aligned to GPW 13. The proposed KPIs are going through various levels of validation; first with the country offices, then with Regional Office clusters and finally with a working group that will assess the KPIs’ alignment with GPW 13 results and other regional instruments to ensure impact at WHO country level in the African Region. The proposed GPW 13 KPIs for WHO in the African Region will then be presented to senior management for approval. These KPIs will assist in: ensuring that the Secretariat delivers on its commitment to the triple billion goal; accelerating progress towards achievement of the Sustainable Development Goals; and delivering impact at country level for improved health development.
20
Going forward, KPIs will be fully integrated into existing processes, systems and structures of budget centres and information technology solutions. Migration to WHO Cloud, including a harmonized web-based internal reporting system, will be further developed to facilitate data collection, analysis and reporting. The Secretariat will continue to be guided by the overarching principle that WHO resources should only be used where there is an expectation of measurable results in terms of improving people’s health, a principle that is highlighted in the WHO Proposed programme budget 2020–2021.5
Every effort will be made to ensure, through the African Region results framework and KPIs, the Secretariat’s capacity to effectively measure and communicate its contribution to Africa’s health. The Secretariat will continue to hold itself accountable for its actions to better serve its Member States to deliver on the “triple billion” goal of the Thirteenth General Programme of Work, 2019–2023 and the Sustainable Development Goals.
Driving impact is the primary focus of WHO’s accountability.
5 The WHO Proposed programme budget 2020–2021 was adopted by the Seventy-second World Health Assembly “the Secretariat will step up leadership at all levels of the Organization to advocate for increased political commitment, drive the agenda in health and lead partners to work together to put health high among global, regional and country priorities;
• strengthen its normative role by delivering the highest quality global public health goods that will make an impact in countries; [and]
bring to bear all its resources, talents, collaborations and networks to support countries in delivering on their priorities for achieving impacts”. A72/4.
Website | www.afro.who.int
