Construction of the OSS Program Performance Index (OSSPI)

The construction of indices to measure “quantitatively complex phenomena”

(Mekong Economics, 2006, p. 14), such as the adoption of the good governance programs, raises important issues (see, for instance, Mekong Economics, 2006).

Indices “are approximations of what they seek to measure” and “have to face issues of aggregation, data truncation and weighting” (Mekong Economics, 2006, p. 14). In order to address these issues, this research strictly follows the recommendations formulated by McCarty (Mekong Economics, 2006. p. 16).

First of all, in order to minimize the aggregation issue, one should apply “similar and transparent aggregation rules to the whole data set” so as not to “leverage the values too heavily through complex mathematical manipulation” (ibid).

As for data truncation, one should avoid the potential risk of overlooking “the differences between groups” and “misrepresent[ing] opinion” (ibid) that can result from defining the ranking of attitudinal indicators in terms of a fixed number of groups (e.g., strongly agree, agree, disagree, and strongly disagree).

Finally, as for the issue of weighting the various indicators, “it involves normative judgments” (ibid). One should proceed by keeping “it simple and to weight the various indicators based on the literature and on experience in the field” (ibid).

The objective of the construction of an aggregate OSS Program Performance

Index (OSSPI) for this research is to complement the presentation of primary data in a quantitative manner. The intent is thus to quantify the effectiveness of the OSS program, (i.e., to measure the gap between the program objectives and the results on the ground, with a quantitative measure that is meant to present this gap easily and clearly). The OSSPI is also available for each locality assessed and for each sub index that composes it.

OSSPI is a synthetic composite measure that is a figure which ranges in value from 1 to 10, lower to higher. The OSSPI is an absolute, and not a relative, index. Localities are ranked on a 10-point scale, where 10 points are given to a locality that has carried out full OSS adoption as prescribed by PMD 181 and by OSS implementation guidelines. In other terms, if a locality scores 10 points, this means that it has adopted all the prescriptions indicated in those documents, and that their adoption has been done completely (i.e., the locality has fully adopted all the improvements concerning the transparency, responsiveness, and accountability of administrative services delivery related affairs; all sub-indices have scored the maximum). For this locality, the degree of effectiveness of the program is therefore the maximum.

While PMD 181 sets out OSS program implementation principles, OSS implementation guidelines are an operational document prepared by the MoHA with the objective of providing guidance to local authorities in the process of setting up and then running OSS. The document sets out responsibilities and tasks that have to be performed by local authorities and civil servants in relation to the implementation and operation of OSS. This includes, for instance, how to set up an OSS venue and its facilities; how to re-engineer working processes for the reception, processing, and delivery of PA services; the range of services to be provided through the delivery mechanism; basic performance management indicators; and other sets of responsibilities.

2.1.1 Methodology for the construction of the index

Given that the questionnaires have been designed in such a way so as to identify systematically to which extent such tasks have been performed, it has been easily possible to formalize each of them into indicators of the OSSPI. All indicators have been grouped into three sub-indices, namely, commune level authorities’ enhanced transparency, responsiveness, and accountability with regard to PASD.

Each one of these three sub-indices is equally weighted as 1/3 of the OSSPI.

Table 18: Sub-indices weightings of OSSPI

N° Sub-indices Value

1 Transparency 33%

2 Responsiveness 33%

3 Accountability 33%

Total 100%

Source: my own representation

Each sub index is made up of indicators that are assessed using one or more qualitative measures (e.g., the qualitative assessment of the answers given by local cadres about a specific topic related to an indicator) and/or one or more quantitative measures (e.g., number of given actions performed by local authorities). The qualitative and quantitative measures are then translated into a figure that indicates the gap between expected results and results observed, ranked from 0 to 10; 10 corresponds to full compliance with OSS legal and operational provisions, 0 equals total non-compliance with OSS guidelines.

All sub-indices, all indicators, and all measures are weighted equally.

Table 19: Sub-indices and indicators

OSSPI

Sub-indices N° Indicators

Transparency 1.1 Visibility of information related to services fees, delivery time limits, and conditions for application of dossiers

1.2 Adequacy of information provided to citizens inside and outside the OSS

Responsiveness 2.1 Adequacy of services delivery to meet people’s needs and expectations

2.2 Lawfulness of charged fees and costs paid by applicant for the delivery of the service 2.3 Financial sustainability of OSS

2.4 Managerial commitment to ensure the performance of OSS

2.5 Effective mechanisms set up by local authorities in favor of citizens for their participation in decisions concerning administrative services delivery related affairs

Accountability 3.1 Local authority commitment to account for their actions in relation to OSS to citizens

3.2 Effective oversight of local PCnls over OSS related affairs

Source: my own representation

Table 20: OSSPI, indicators and measures

▪ The quality of the information posted;

▪ The feedback of OSS clients that were asked if services fees, delivery time limits and conditions for application were clear to them;

1.2 Adequacy of information provided to citizens inside and outside the OSS

▪ Whether local administration has ever carried out at least one information campaign in the villages;

▪ The feedback of OSS clients that were asked whether the information provided outside the OSS venue concerning OSS related affairs was adequate;

▪ Whether local officials have displayed in the OSS venue information related to recently issued legal regulations, complaint and denunciation procedures, local budget or information related to the running projects financed by local budget

▪ Since the opening of OSS, one of the following elements has been adapted at least once:

- OSS timetable;

- the frequency of services delivery;

- the range of services delivered;

- the procedure related to the transfer of the dossier to the higher administrative echelon;

▪ Feedback of OSS clients that were asked if the delivery modalities of OSS were in line with their expectations;

2.2 Lawfulness of

▪ Whether official services fees charged to applicants were identical to those set by the Ministry of Finance (MoF);

▪ Whether services whose fees were not regulated by the MoF were officially provided free of charge;

▪ The feedback of OSS clients that were asked if, since the implementation of OSS, they had noticed a change in the behavior of civil servants with regard to their disposition to extract extra money from clients;

2.3 Financial

sustainability of OSS

▪ At least 25% of the proceeds generated by OSS re-invested in the OSS;

2.4 Managerial commitment to ensure the

▪ Qualitative assessment of the corrective actions taken and suggestions for improvement formulated by local cadres in relation to OSS performance;

N° Indicators Measures

performance of OSS ▪ Qualitative assessment of the collaboration between OSS staff and the Heads of functional departments with regard to OSS management;

▪ Whether lessons learned were shared with upper level competent agencies with a view to setting up a national database;

▪ Qualitative assessment of the mechanisms set up by local authorities in order to include citizens in decision making concerning:

- the range of services that should be added to the OSS;

- the delivery modalities of OSS (e.g., timetable, the frequencies of services delivery, etc…);

- other issues related to OSS adequacy to clients’ needs;

Accountability sub index

▪ Qualitative assessment of the answers provided by local officials concerning how illicit bureaucratic practices are prevented, managed, and solved;

▪ Qualitative assessment of the modality chosen by local authorities to notify clients when a delay in the delivery of the service occurs;

▪ Qualitative assessment of how local authorities manage people’s complaints and denunciations with regard to administrative service affairs;

▪ Feedback of OSS clients that were asked if, in case of wrongdoing, they would trust that local authorities would effectively take action against a guilty bureaucrat 3.2 Effective oversight of

local PCnls over OSS related affairs

▪ Qualitative assessment of the capacity of local officials to mention the concerns and expectations of the members of the PCnls vis-à-vis the results and the performance of OSS Source: my own representation

2.2 Presentation of the findings: assessment of