Université Libre de Bruxelles Faculté des Sciences
Institut de Gestion de l’Environnement et d’Aménagement du Territoire
Indicators for Sustainable Development:
A Discussion of their Usability
Thèse présentée par
Tom Bauler
En vue de l’obtention du grade académique de Docteur en Environnement Mai-Juin 2007
Directeur : Prof. Edwin Zaccaï (ULB)
Jury : Profs. Hans Bruyninckx (KU Leuven), Walter Hecq (ULB-CEESE), Frédéric Varone (Université de Genève), Philippe Vincke (ULB), Edwin Zaccaï (ULB)
Acknowledgements 7
Introduction 9
Chapter 1 ‘Indicators’ and ‘Indicators for Sustainable Development’ 15
1.1 Indicators for Sustainable Development 18
1.1.1 The relationship between indicators for sustainable development and assessments 18 1.1.2 ‘Indicators of sustainable development’ or ‘indicators for sustainable development’? 19 1.1.3 Incremental versus structural levers to policy change 21 1.1.4 Indicators and the self-generation of Sustainable Development 23 1.2 Characterizations of Indicators for Sustainable Development 24
1.2.1 Historical backgrounds 25
1.2.2 Definitions 28
1.2.3 Types and typologies of indicators 31
1.2.4 Typologies of indicators: some examples 42
Conclusion to the chapter 45
Chapter 2 A Procedural Understanding of Sustainable Development:
Principles and Processes 47
2.1 Sustainable Development: Systems’ Approaches and Processes 50
2.1.1 Systems approaches 52
2.1.2 Processes 59
2.2 Translating the ‘Projet de Société’ into Principles 62
Conclusion to the chapter 65
Chapter 3 Sustainable Development and Assessment 67
3.1 Decision-making and Information: Attempts to assess Information 74
3.1.1 Decision-aiding and rationalities 78
3.1.2 Handling information in decision spaces 83
3.1.3 Towards a generic model for information assessment 92
3.2 SD and Evaluations: Principles of Sustainable Development as Limits to
the Utilisation of Assessments 99
3.2.1 Multiple Dimensions : L,C,S-criteria and the principle of ‘integration’ 100 3.2.2 Participative Assessments : L,C,S-criteria and the principle of ‘participation’ 105
3.3. Approaches towards assessing Influence of Indicators for Sustainable
Development 110
3.3.1 Selected approaches towards assessing ISD-use 111
3.3.2 Legitimacy, Credibility, Salience at the level of ISD 123 3.3.3 Linkages between the L,C,S-framework and the approaches for the
assessment of the indicator influence 135
Conclusion to the chapter 139
Chapter 4 Institutionalisation of ISD : a major Factor to characterize the
Usability of Indicators for Sustainable Development ? 141
4.1 Institutionalisation and ISD 144
4.1.1 Institutionalization and institutional embeddedness 145
4.1.2 Limits to the institutionalization of ISD 150
4.1.3 The L,C,S use-criteria facing the institutionalisation of ISD 153 4.2 ‘Institutional Embeddedness’ as a second Axis to the Assessment of
ISD-Usability 156
4.2.1 Indicators and their institutional embeddedness: a matter of
‘boundary organisations’? 156
4.2.2 Steering ‘boundary organisations’: ‘reflexive governance’ applied to ISD 158 4.2.3 Integrating ‘Institutional Embeddedness’ to the L,C,S-framework 160
Conclusion to the chapter 166
Conclusion :
From ‘proceduralism’ to ‘institutional embeddedness’ to ‘reflexive institutionalisation’:
a prudent outlook on enhancing the usability of ISD (and more?) 169
References 175
Acknowledgments
It is as always very difficult to find the correct, not too clumsy, anchor for the acknowledgments of such an enterprise as a thesis. I don’t excel in this type of exercise.
Of course, I owe particularly to my parents without whose sincere enthusiasm to see me evolve in research and higher education, it would have been much more difficult for me to find the motivation to invest in this enterprise. In a very different dimension, and of course in many uncountable respects, I am deeply indebted to Sara, the other stable, continuous pillar in my life (in parallel to an unfinished thesis project). I owe very much also to those of my very, very good friends, who made me laugh at the surrealism of my own condition every time I met them.
Not strictly in parallel to these more personal than professional acknowledgments, I am very appreciative of Edwin Zaccaï, director of the present thesis, main cause of it being completed today, and very much a reference-person to me. Particular credit also to Walter Hecq, without whom I wouldn’t have entered ULB, or the study field of ISD.
Big waves to all the others, who were more or less involved in the present enterprise, and be it only because they were so polite not to ask me to explain them my work.
Introduction
Context
After nearly 20 years of existence as a policy referent, the effective translation of Sustainable Development (SD) into policy processes remains a matter of debate, even of experimentation, as new policy initiatives are still being developed. For instance, precisely at the moment we complete the present effort, mid-April 2007, the Belgian parliament is taking decision to include an article on SD into the Belgian constitution, whereas parts of the federal Belgian administration are triggering a debate on the necessity and opportunities to institutionalise a long-term participatory planning process for SD.
Obviously, since the Brundtland-report and the Rio-summit there have been shifts in the interpretations of SD. Over the years there have also emerged different - and sometimes innovative – approaches to proceed with the operationalisation and implementation of SD into public policy processes. Two generic, parallel stances have been followed. On the one hand, one observed the emergence of specific SD-policies and notably of SD-strategies, which meant to provide a top-down reference point for policy-makers. On the other hand, a handful of specific instruments, processes and tools were initiated which should assist to mainstream the translation of SD-criteria and principles into policy processes by influencing the configuration of policy moments such as public policy evaluation and assessment, policy communication, policy formulation…
Within this second – non-programmatic – approach to translate SD into policies, the choice of the instruments, processes and tools to be preferred and promoted appears to have considerably changed over the last years. In effect, right after the Rio-summit a wave of processes at national, global, local, urban… level were initiated, which were concerned with the construction of indicators for sustainable development (ISD). Before the turn of the millennium, nearly every country in the developed world had its ISD-initiative either accomplished or in the pipeline. Besides, hundreds of local communities had stepped into their ISD-processes. And private corporations had accomplished a first version of international standards for ISD-processes at the level of corporations.
Today, many of these initiatives have weakened, both in intensity and number. Even if some countries still publish yearly updates of their SD-reports based on ISD, and some initiatives finally concretize their processes with a first ISD-report (e.g. Eurostat’s SDI), hardly any notable ISD- processes or approaches seem to have emerged for some years now. In parallel to this diminished interest, or maybe because of it, or maybe as a response to the initial explosion in ISD, the singular question of the utility of ISD has appeared. In fact, many observers and supposed users of ISD had problems to detected what would have justified the relatively intensive investments in resources and hopes which have been put on ISD. ISD did not seem to be able to live up to expectations; many of them simply didn’t deliver the awaited effects and seemed to be quite strictly discarded by their supposed users. Of course, there could be many reasons for this, and one obvious one being that the expectations were not realistic. In any case, the question surfaces among researchers and policy actors alike, and has some pertinence outside of the debate on ISD: evidence-based policy making is becoming the general paradigm in public management, and debating the utility-mechanisms of
information, indicators, assessments… could help to understand the operationalisation of ‘New Public Management’ in the realm of SD.
The present thesis wants to be a contribution to this debate at the level of ISD.
Research questions
As in many thesis-processes, our research questions evolved during the exercise. At the beginning stood the generic question to analyse and understand the impacts of ISD on decision-making. And more particularly, to derive thereof a series of construction- and configuration standards, which could contribute to enhance the impacts of SD-assessments and of ISD in policy-making processes: What are the success factors that characterize the impacts of ISD on policy-making, and how can these be translated into construction-criteria ? This initial question underwent a double refinement.
First, it appeared to be more accurate to conceptualize the link between ISD and policy-making in terms of utilisation, instead of impacts, because utilisation could better account for the multiple diversified uses which can be made of information in policy situations. Considering the question at the level of the utilisation of ISD appeared however to be a too large conceptualisation when engaging into a debate of ISD-performance. In effect, Utilisation can be sequenced into a series of sub-concepts, e.g. can be expressed in terms of information usability, information accessibility, information digestion, information use by policy-makers, information influence on policy-makers, information impact on the policy decision. Because it was important to us to focus on those aspects which allow to consider ISD as science-based decision objects, and thus to feed our discussion back to the characterisation of ISD-processes, so to say to their configuration, we focused on those aspects of indicator utilisation which are directly evolving from ISD as objects. We focused our analysis thus to the first element in the utilisation-chain, namely on usability, i.e. the potential of ISD to integrate decision processes.
Second, the quest to translate the results of the discussion of indicator utilisation towards the creation of construction criteria and standards quickly revealed some fallacies. As one can expect, indicator utilisation is far from being a linear process. Utilisation can happen without apparent reason or even without an identifiable source of information, it can be considerably delayed, it can be opposite to the expected… Indicator utilisation depends also on the context of the decision process, the configuration of the decision processes, the articulation between the decision actors. Assuming that it would be possible to create, even prudent and non-normative, ISD construction criteria which would effectively guide influence, would however imply an implicitly mechanistic comprehension of utilisation. We focused thus the analysis towards the level of the characteristics of ISD-processes, and how these characteristics link to the context of the decision; i.e. in a certain sense, how usability can be linked to the decision context.
From these two focalisations of the initial research question, a double interrogation finally emerged to guide the analysis:
Which are the characteristics of ISD-initiatives that are influencing the usability of ISD in decision situations? And: Can we identify a key which allows to read and analyse these characteristics, i.e.
construct the usability-profile of ISD-processes, with respect to the configuration of the decision situation?
Methodology
The present thesis is organized as a discussion. Consequently, we will present a series of arguments that will be highlighted from different perspectives. The arguments presented - as well as the perspectives that allow their discussion - originate from analyses of ISD-processes, but even more so from a thorough literature study. In effect, rather wide fields of disciplines - and thus literature - can be mobilized to take an insight into the research questions, and as a consequence the literature used presents an interwoven pattern of disciplines, research questions and research domains (see figure 1).
Figure 1 – Visualisation of the literature-study (partial) Evaluation
Sustainable Development Procedural Perspectives on Sustainable
Development
Evaluation Utilisation
Information processes in Sustainable Development Indicator Utilisation
Evaluation Use
Environmental Decision- aiding Instruments Indicators for
Sustainable Development
Policy Processes Utilisation of
Indicators for Sustainable Development
Contrary to this plethora of anchors in literature, literature that could be of relatively frontal and direct use to our research questions is fairly rare (except for Boulanger 2006; Hezri 2006; Ortega- Cerda 2005; Gudmundsson 2003; Rosenström 2002 and ‘in press’).
Facing such a variety of literature domains and disciplines, our present effort is striving to be inter- disciplinary, in a sense that we transpose the original literature and its concepts from their original, often disciplinary fields of research onto our own, while enlightening them with perspectives and arguments taken from other disciplines and researchers. However, our approach faces a structural limit, because per definition inter-disciplinarity can only be simulated if undertaken by a single researcher, as its operationalisation necessarily calls for different disciplinary actors to interact and produce a new level of analysis.
Organisation of the thesis
Because of the multiple overlapping literature perspectives and analyses, which we used during the thesis, there is not a unique rationale which could have been applied to structure the work.
In order to discuss the first level of our research questions, i.e. which are the characteristics of ISD- initiatives that are influencing the usability of ISD in decision situations?, we investigate in the first place the interpretations which can be given to our object of study. Chapter 1 is thus concerned with exploring definitions, configurations and particularities of indicators for sustainable development.
We investigate to what extent indicators can be understood as levers for policy change, how they are organized with respect to processes of sustainable development. Finally, we present different methodological constructions of ISD, and illustrate a series of typologies to bundle and structure the different approaches to ISD.
Among the different perspectives which can be taken on ISD, one of the more obvious ones is influenced by the interpretation to be given to the object of the assessment, i.e. the interpretation of Sustainable Development (SD). Chapter 2 presents thus a synthetic excursion into the possible conceptualisations of SD. We identify three different approaches to illuminate SD: the systemic character of SD, the procedural understanding of SD and the normative-political comprehension of SD in terms of a collection of principles. Rather then indicating antagonistic approaches, all three approaches can be interlinked at the level of indicators for sustainable development.
The first and the second chapter are basically needed to restrict the signification of ISD and to investigate the policy domain they are meant to operate in. Subsequently, we penetrate the heart of the problematisation.
Chapter 3 is concerned with elaborating an analysis of the linkages between SD and evaluation use in the case of ISD, with the objective to identify those characteristics that co-define the usability of ISD in policy making. In a first place, we investigate the fact that the operationalisation of SD, through its dynamic and procedural components, necessarily calls for assessments. Consequently we take a step
back, and explore the linkages between public policy and decision making in general, and the information flows that sustain these processes. We will identify a collection of assessment- characteristics, which have been developed to describe the mechanics of the assessments’ integration into policy making. We will discuss on the basis of these characteristics, how far SD, and more precisely some of its main defining principles, can be acknowledged to be generically counterproductive to the production of usable assessments. Subsequently, we discuss the interrelationships between ISD and the identified asssessment characteristics.
Chapter 4 will discuss a further axis of analysis to the usability-characteristics of ISD. The preceding analysis and discussion of the characteristics could not satisfy our second research question, i.e. can we identify a key which allows to read and analyse these characteristics, i.e. the usability-profile of ISD-processes, with respect to the configuration of the decision situation?. By confronting the usability-characteristics to the organisation and proceduralism of ISD, i.e. their institutionalisation, we will propose a second axis to the prior established usability-characteristics. This supplementary level of reading the usability of ISD wants to be a step towards providing a structure for the management of usability.
Chapter 1
‘Indicators’ and
‘Indicators for sustainable development’
Indicators are still at the heart of the debate on sustainable development, whatever the level or stance taken: sectoral issues (e.g. transport&environment; climate change; greening of public procurement…) are claiming to use and develop indicators as well as global, multidimensional issues (e.g. the monitoring of the ‘Millennium Development Goals’). State-of-the-Environment reporting on country level is inextricably linked to the use of indicators, as is the implementation of the ‘Global Reporting Initiative’ on the level of firms, which issued its 3rd version of monitoring principles.
Etcetc
Indicators get developed by some in order to help them define their strategies, whereas others assess the success of their strategy with them. Indicators are used to evaluate and communicate on the performance of buildings and construction sites, as they are used on the level of urbanism. Indicators are initialized for small-scale evaluations of public space management or the allocation and use of local development funds. Simultaneously, indicators are used to communicate on large scale ex ante
‘Sustainability Impact Assessments’. Sustainable indexes are developed to rank stock portfolios and pension funds. Academia is striving to discuss composite indicators, which are supposed to replace or complement Gross Domestic Product (GDP) in the near future, whereas adaptations of the same GDP to integrate environmental and social variables are meant to keep the economic aggregate at live.
Sometimes indicators seem to be mere by-products of data treatments, such as maps extruded from the latest GIS-software. Sometimes indicators represent the emerged part of an empirical calculus, as it is the case with communicating the outcome of extensive Life-Cycle-Analysis (LCA). On other occasions, indicators are condensing the results of complex and time-consuming data-collection and –structuration efforts, as it is the case with the attempts to green the national accounts by constructing satellite environmental accounts. Lately indicators are outputs of ‘Sustainability Impact Assessments’
as well as of processes using ‘Multi-Criteria-Decision-Analysis’. Sometimes indicators are nothing else than a more or less successful combination of reheated data existing elsewhere in administration.
Obviously, what is so harmoniously called ‘indicators for sustainable development’ (ISD) cannot be related to a single well-defined object. The multitude of initiatives and perspectives referring to indicators in the context of sustainable development is rendering an ambiguous and heterogeneous picture of the object. This first chapter is thus concerned with a general presentation of ‘indicators for sustainable development’ (ISD).
For the sake of internal coherence and understanding of the following chapters, we need to clarify some notions: it is the multiplicity of possible understandings about what an indicator is, which forces us to present as clearly and as unambiguously as possible our view on indicators. While stressing and exploring the diversity of the object of study, we intend to distil some general features of ‘indicators’, which will help us to develop useable working definitions of different types of indicators. The aim here is to present what could be named the dénominateur commun, e.g. the minimal characteristics between different interpretations and conceptualisations of ISD.
In the first section, we define indicators, and present the contexts into which these instruments inscribe themselves. In a second section, we characterize more specifically ISD from their historical base, their appearance, their definitions, and their methodologies. We sketch also a selection of ISD- typologies in order to account for the diversity of existing ISD approaches. Finally, the third section will explore the roles which ISD are assumed to fulfil as it appears in general literature on sustainable
development (SD) or ISD. This last section will be of particular interest, because it introduces to the heart of the present work, i.e. to provide a contribution to the better understanding of the levers for ISD in decision processes and policy making, hence discuss how far these roles and expectations could be met with ISD.
Subsequently the use of the term ‘indicator’ refers to ANY type of indicators (including indicators for sustainable development): we use it as the generic term. ‘Indicator’ is thus NOT to be taken as a short form for ‘indicators for sustainable development’, which will be addressed in short solely as
‘ISD’.
1.1 Indicators for sustainable development
Before presenting a working definition for ISD (see 1.2), the following paragraphs merely introduce to a number of observations, which are regularly made when encountering indicators and/or ISD.
It is often assumed that the performance of ISD is decided upon at first sight: as indicators synthesize information in a way as to render it comprehensive for a large number of users in a short time period, it is their potential to be read and understood quickly, which is of importance when assessing the performance of ISD.
If a more thorough usability framework will be developed later (see chapters 3 and 4), the aim here is simply to approach indicators and to account for a series of ambiguities raised by our object of study:
What should or can be seen when approaching ISD? Is it possible to distinguish them from other types of policy initiatives or policy domains that refer to indicators? How far is this necessary? Are ISD to be differentiated from indicator assessments in the context of SD?
1.1.1 The relationship between ISD and assessments
A first ambiguity to be encountered when being concerned with ISD, stems from the relationship and hierarchy between ISD and assessment (or ISD and evaluation); as a matter of fact, ISD can either be products of an assessment, or the development of ISD can be the basis for the assessment.
When indicators are developed as products of assessments, they are rendering the results of an assessment or evaluation or monitoring exercise. In these cases, it is the methodology used for the assessment, which mainly induces the choice, quality, perspective and coherence of the produced ISD. Life-Cycle-Analyses (LCA) are good examples in this regard as indicators are used in LCA to synthesize extensive data-gathering and –treatment into an easily readable and understandable message. However, the definition of the indicators, the data used, the definition of the system boundaries… do not depend on methodological concerns about indicators. The parameters of the assessment are defined by the LCA-methodology and indicators are merely used to present the output
data of the LCA, and to facilitate the comparison of results, which in this case is the ranking of products and services according to their environmental consumption. The quest for coherence - both methodological and on the level of the individual indicator - is left with the methodology of the assessment. In other terms, the indicators’ methodological quality is dependant of the quality of the assessment methodology. In turn, in our example, all the flaws and uncertainties of LCA are injected also to the indicators.
On the other hand, when indicators are at the core of the assessment, the concern for methodological coherence and robustness is rooted on the indicators’ level: the coherence between the indicators as well as the robustness of each indicator is gaining importance. Among the numerous examples, the most obvious ones of this type of assessment are the ISD-lists developed by the UN’s Commission for Sustainable Development (2001) after 1995 and which were meant to participate to the assessment of the implementation of the Rio-signed Agenda21. Lately, at international level, the same type of indicator-based assessment mechanism was introduced for the Millennium Development Goals. Other examples of this type of assessments based on indicators are the numerous State-of-the-Environment reports on international or regional scale (EEA 2005) as well as on national scale: number of these assessments are developed along methodological frameworks, such as DPSIR-indicator lists (see section 1.2.3), which stress the fact that the coherence of the assessment becomes organized from the level of the indicators.
Arthur Lyon Dahl (personal communication, 2004) introduced a slightly different distinction between these two fundamentally different starting points for the development of ISD. He labelled the approaches “assessments with indicators” (i.e. indicators are by-products) and “assessments based on indicators” (i.e. indicators are the core of the assessment). Dahl points also to the fact that if both approaches co-exist within the SD-community, there are only very few occasions where both approaches converge methodologically. Consequently, the results obtained by either approach will differ largely, because the choice of perspective prescribes the setting of the system-boundaries used for the assessment, and because boundary-setting is of predominant influence.
Both relationships between ISD and assessments can be equally important in their contribution to a SD-process, be it in order to develop a more nuanced image of the relationship between humans and environment, or to present nuanced assessments to decision-makers. Conversely, differences between the two approaches to ISD are more profound than is apparent at first sight, especially with regard to the utilisation that can be made of them.
1.1.2 ‘Indicators of sustainable development’ or ‘Indicators for sustainable development’ ?
A second ambiguity when treating indicator initiatives stems from the usage of terminology, and can be illustrated with a detour at semiotics. ISD are either translated as ‘indicators for sustainable development’ or ‘indicators of sustainable development’. If some rare authors seem to use both terms
(or any of the terms) without paying attention to the implicit differences they convey, these differences exist and reflect a fundamentally different understanding of the object under assessment.
Assessing a situation with the use of ‘indicators of sustainable development’ implies indirectly that sustainable development has been defined as a precise object (in some cases, even as a policy target).
In such an understanding, ISD are then configured with respect to the object, e.g. the distance to the objective state. We will discuss later (see chapters 2 and 3) considerations about uncertainties and indeterminacy of SD, but for now it should be acknowledged that SD is a dynamic process of social transformation. SD cannot be translated directly into a series of parameters, which would define a state of SD per se.
For instance, extensive knowledge about carrying capacities would be needed if we intend to define SD as a target-state. As these are largely ignored today, as well as impact functions and causalities for most of the interlinkages between environment-economy-society, we can identify at best a set of principles (see chapter 2) that delimit the terrain of SD. At the same time, using this wording conveys a message of ISD being specifically developed and solely applicable to SD, which is rarely the case with the indicators composing ISD-lists or ISD-composites. Rather is it the way of composing (process, objective, utilization…) the lists or the aggregates, which are specifically influenced by the SD paradigm, but not necessarily the indicators therein.
In parallel, in the case of referring to ‘indicators for sustainable development’, the uncertainties attached to SD and the inherent impossibility to clearly define SD as an operational target in terms of a state, pose less problems. Indicators are simply meant to participate to an assessment, which is operated with reference to the socio-political process of SD; indicators are meant to contribute to the apprehension of the pathway to SD. This pathway towards SD can, on the contrary to SD as a state, be constructed, for instance by contradiction: even if we don’t know exactly the endpoint of the journey, the situation of today comprises a number of socio-environmental evolutions that can be identified as unsustainable. Indicators could be constructed to monitor these evolutions.
Formally speaking, and in our context here, we prefer to speak of ‘indicators for sustainable development’, even more so as this wording underlines the indicators’ competence to operate both as a lever and as an instrument for the process of accomplishing and working towards SD.
Other ambiguities exist at the level of terminology, as ‘indicators for sustainable development’ is not the sole wording used to describe ISD. For instance, ‘sustainability indicators’ is used too (for instance, Bell and Morse 1999), but its reference to the more holistic ‘sustainability’ concept (for a comment on differences between ‘sustainability’ and ‘sustainable development’, see for instance Robinson, 2004) makes indicators developed thereof mostly eco-systemic in nature and discipline.
1.1.3 Incremental versus structural levers to policy change
Necessarily, ISD-initiatives reflect the understanding of their authors with regard to the nature of necessary changes in decision-making that will lead to SD. As in the SD-community different shades of green, red and blue tendencies coexist and occasionally overlap, there is no common understanding neither on the levers that ISD have to contribute to. Basically, we want to distinguish 2 approaches at this level: incrementalism and structuralism.
Incrementalism
In an incremental understanding, decision-making for SD is a matter of integrating and redirecting mostly existing knowledge so as to allow decision-makers to re-evaluate the relevance and importance of the constraints and opportunities they are taking into account. In this sense, incrementalism in a context of SD-policies is often understood as a matter of readjusting the prominence of economic rationalities in public decision-making by promoting the recognition for instance of environmental limits and negative social impacts. ISD are conceived as being tools that help decision-makers to take notice of these potential impacts of and limits to their decisions: from a mere mono-dimensional (administrative and hierarchical) decision-making, ISD want to render accessible multi-dimensional information for decision processes. Incrementalism poses thus that ISD insert themselves into the conditions and processes decision-makers find themselves in: time and budget constraints, inappropriate and incomplete knowledge of issues that lie outside of their usual sphere of influence, concentration to their direct spheres of influence discarding for instance long term and distant impacts, technocratic and expert-driven decision-support… Acceptance of incrementalism has a number of consequences on ISD as an object in policy making, such as for instance :
be rather few in number (e.g. comprise a number of headline indicators as in the case of the EUROSTAT-ISD) and as aggregated as possible (e.g. see the Environmental Sustainability Index): decision-makers are already overwhelmed by traditional information sources. Any additional type of information which wants to be grasped by decision-makers has to be very concise;
strongly simplify interactions and messages (e.g. as is done with the Ecological Footprint): decision-makers are assumed to have very limited amount of comprehension, time, capacities and interest that they can invest into the numerous emerging issues of modern society;
be directed towards influencing the configuration of policy outcomes, e.g. by promoting potential levers of action (e.g. use decoupling indicators to promote policies of eco-efficiency).
Incrementalism is based on the idea of accepting existing reference points and mechanisms in policy- making. ISD are meant to provide additional, more diversified information on emerging policy issues
and domains, but stick to the prevalent norms and conventions when it comes to the configuration of the information and its integration into policy processes. Ziegler (op. cit. : 167) posed the challenge in terms of making “a different type of development ‘measurable’ in a similarly reductionist mode as that of the ruling paradigm which provides politics with metaphors encapsulated in simple numbers”.
Structuralism
Structuralism takes a different stance. Acknowledging that the ‘bad’ (e.g. in terms of natural resource depletion, or decrease of biodiversity…) performances of modern societies with regard to environmental and social criteria are rooted in inadequate decisions taken on the basis of inadequate information in inadequate decision-contexts formed by inadequate decision-processes, implies to recognize the necessity to reform the entire decision-making system. ISD are seen in this understanding as means and levers which contribute to reprocess that decision-making system.
Associated claims for transparency of decision-making processes are additionally acknowledged to be met by rendering underlying, hidden information flows and contents (i.e. a sort of meta- information) accessible and transparent. ISD allow citizens and pressure groups basically to gain access to the same level of information than institutional or political decision-makers.
Simultaneously, the claim for more participative decision-making is met by allowing these stakeholders to take influence on the design, structure and content of the ISD. The change in the power-relations at the level of the information configuration and access will induce an equal change in the power-relations in general. The one who participates to the information construction, is the one who participates to the decision. The impacts on the level of ISD are then to:
Adjust ISD as close as possible to ‘reality’, i.e. use holistic representations of the systems under scrutiny, and don’t refrain to integrate, if appropriate, theoretical and conceptual notions related for instance to systems’ theory. Present unambiguous facts and data. It is up to the decision-maker to extrude potential action and policies from the information that is made accessible to him. It is thus not necessary to participate to the interpretation of the data presented, or even to link ISD exclusively to policies.
Don’t limit artificially the number of indicators. Complex issues cannot be represented with a handful of variables. As a consequence, instead of accepting that decision-makers use few information, they need rather to be trained to gain the capacities to process larger amounts of more diversified information or as put by Ziegler (2002 : 168), “(…) the core of the discussion about sustainability is in learning to understand the world in its complexity rather than simplifying it”.
Both incrementalism and structuralism have major flaws. Grossly: incrementalism is not accepting that the current decision-making system and the information treatment it induces are neither perfect, nor are they unchangeable. Structuralism, on the other hand, oversimplifies and uses as a basis of
their thinking a too naïve picture of current decision-making. The challenge lies probably in reconciling these approaches.
1.1.4 Indicators and the Self-generation of Sustainable Development
A further ambiguity of ISD is stemming from their unclear relationship with the process of SD itself.
The initiators of the ISD debate at the international level had a very precise idea of the linkage between ISD and SD, namely "(…) indicators of sustainable development need to be developed to provide solid bases for decision-making at all levels and to contribute to a self-regulating sustainability of integrated environment and development systems." (§ 40.4) (UNCED, 1992). The same type of understanding of the relationship between indicators and SD appears to be widespread at the local level: at the level of Local Agenda 21, ISD became a major function in generating and sustaining the process. Logically, the quality of the SD-processes, independently of the institutional level, depends to a certain extent on the potential and performance of some form of SD-Plan or Implementation Strategy. The delivery of such strategies depends partly on the mechanism that was initialized to measure and evaluate the distance to the targets or the evolution of the trends that the strategies appointed. Simultaneously, the initial picture and understanding of the existing situation needs often to be fine-tuned in order to be able to derive the desirable targets and trends, which calls again for the development of some form of measurement system. Both of these needs, the initial pre- strategy picture as well as the ‘continuous’ evaluations of the situation, are reputed to be satisfied with ISD (at least to a certain degree).
At the same time, we should not forget that ISD and SD pay tribute to their ‘époque’, the nineties, where all of a sudden it seemed possible to provide access to us world citizens to a large amount of background information. The information society made a big leap ahead and its products seemed to become the keystone to improve the living conditions of the masses (whether in the North or the South): access to information was being held a direct proxy for access to power and to decisions. The only visible hurdle on the way to a generalized empowerment through information - but which seemed technically manageable all of a sudden - was to facilitate access to that vital information. ISD were supposed to participate strongly to render the necessary accessibility: they are simplified, communicable, distributed, synthesized information.
If most authors and institutions have become more careful since then, the underlying power of indicator evaluations is still widely acknowledged as being an important part of SD-processes. But, even very enthusiastic and rational-minded institutions ask today whether it should and could be the indicators that could pull the whole process of SD, as was underlined at UNCED in 1992.
The few issues we addressed here are not exhaustively reporting the numerous ambiguities with ISD at all. As said before, these first pages were rather meant to hint at the most direct interrogations that can be raised when having a first look behind ISD. Continuously through the rest of the text we will
come back thus to one or the other of these questions and develop them further, not necessarily to present answers or portray best-practice and solutions to any of them, but with the aim to see how far we can deconstruct these ambiguities and subsequently reconstruct them into a series of underlying and overarching questions. Obviously, the deconstruction and analyses we intend to provide take a utilitarian turn: the central question we identified to all of these issues is the one of the utilisation of ISD, and in a wider perspective, it will be the impact mechanisms of ISD that we want to shed light on.
In order to achieve this, we could also have used a precise decision-making model, which would allow us to monitor in a stricter sense the points of influence of ISD. Typically, using such an approach would have meant to sequence the text into: here is the model; these are the observations we made; show how the observations can be explained by the model, and see if the deviations from the model call for a new model. We could have selected and adapted one of the decision-process models that exist in literature, and use it to describe deviations or compliance between the model and what we interpreted from observed use of ISD. If such was our initial intention, it appeared that our object of investigation – i.e. ISD - was too slippery and ambiguous itself to be conveniently tested upon such compliance. Especially as the hypothetical selection of a model is already rendered sufficiently complicated by the inextricable complexity and vagueness of SD. Selecting and using a model would have meant to delimit SD with an operational definition, or in other terms, it would have meant to abandon a considerable part of the concept’s attractiveness in order to seal it into a manageable and managerial question of public choice.
The subsequent work remains thus utterly unfinished in a sense. It lays only a first brick to a construction of a wider model of ISD-use. On the other side, the advantage of the following is to present a number of quite independent analyses.
However slippery and ambiguous ISD are, in the following of this introductory chapter we first portray, interpret and contextualize a series of definitions of ISD. Apart from the fact that these reveal that there are many different accents possible when qualifying ISD, it appears that there are just as many different ISD: our object of study is completely heterogeneous not only from the understanding that different people have of it, but also from the very point of view of its basic characteristics.
1.2 Characterizations of Indicators for Sustainable Development
There are many different possible approaches to define indicators in conjunction with SD, and perhaps the best would be to refrain from doing so. Without reference to a clear context or precise policy-situation, it appears that “attempts to define the characteristics of indicators per se are not helpful” (Bosch, 2002 : 77). Obviously, such desertion from defining ISD cannot in turn be helpful at all for our enterprise. We analyse in the following some of the more conventional definitions and characterizations of indicators in the context of SD in order to circumscribe the meanings of ISD.
First we explore the historical context of ISD, then we stipulate and discuss a definition and finally we explore some of the existing types of ISD.
1.2.1 Historical backgrounds
If SD is a rather contemporary ‘paradigm’ or policy domain, indicators in the sense they are defined under SD have a historical background. At least since T.R. Malthus (1766-1834) or J.S. Mill (1806- 1873), questions with respect to measurement and monitoring of societal development are occasionally in the centre of the policy debate. One of the latest such episodes at international level was the public and expert-debate in the mid-1990s around the international UNDP-measurement of Human Development (Sen, 1999).
More generically, ‘Public choice’ has always been a matter of monitoring, and thus valuing, societal evolution and development against criteria defined by individual or group preferences1. In this sense, ISD have ancestors, and three episodes seem of major importance to characterize ISD:
the development of the System of National Accounts (SNA)2 in the 1940ies the social indicators’ movement of the 1970ies
the formalization of environmental policy performance indicators since the 1980ies.
System of National Accounts (SNA)
The intellectual initiation by the USA, and the subsequent world-wide (in the context of Bretton- Woods) use and standardisation of a system of accounting at national level, was intended to permit monitoring monetary flows and stocks. National accounting, and thus SNA, is at the basis of the macroeconomic indicators we use today, the most prominent being Gross Domestic (or national) Product (GDP/GNP). While any economic debate based on analysis largely owes to the System and its ongoing developments (e.g. PPP – ‘Purchasing Power Parity’ is one of its younger derivates), it also inspires a wealth of people who oppose either the underlying economic thinking, or who do not accept the supremacy of the indicators derived. A number of improvements have thus been proposed through the years, notably in order to redefine the approaches of measuring development.
As GDP / GNP was perverted through the years into a proxy measure for well-fare, well-being or societal development, it triggers a number of ISD-approaches that intend to find replacement indicators that would more accurately assess issues of non-economic development and societal progress (e.g indicators of happiness or quality of life). Along the replacement of GDP / GNP,
1 For a brief introduction into the history of ‘public choice’ and the articulation of ‘public choice’ as a fundamental economic question: Saint-Upéry (1999).
2 The current system, updated from SNA1958 and SNA1973, is formalized in the SNA1993 nomenclature developed jointly by the main international organisations. An overview and in-depth knowledge can be accessed via the UN – Statistics Division’s Internet Data-base at http://unstats.un.org/unsd/sna1993/introduction.asp .
number of ISD-attempts make efforts to methodologically adjust GDP / GNP (e.g. Index of Sustainable Economic Welfare or Genuine Progress Index) so to integrate in its calculus the major types of flows that are acknowledge to keep GDP from being a measure of welfare: 1) issues that generate monetary flows, but are universally understood as negative contributions to human welfare (e.g. pollution, accidents…); 2) issues which are not taken into account in the classical GDP calculus but have a positive economic value (e.g. informal trading sector, household keeping).
However, as the System of National Accounting is undeniably a successful attempt to standardize measurement, the national accounting approach is also used as blueprint for ISD which attempt to account for complementary, non-financial flows: environmental accounting as well as social accounting are both under development. Especially environmental accounting3 shows promising developments, and indicators derived thereof are starting to be used as complementary information to economic indicators. As logical derivates, their complementarities to economic accounting yield a very interesting capability for their use in policy-making.
Social indicators’ movement
Indicators are very much linked since their origins, to monitoring and assessing social phenomena and policy responses. The evaluation of a society’s achievements - as well as the distribution of goods and bads among society - took however different turns according to the ‘époque’ and scientific evolutions. During the 1970ies, issues of social well-being were somewhat prioritized in public policy making, exceeding the popularity of an economic reading of well-fare. Consequently, researchers and public authorities saw a wide interest in improving the construction of indicators of social interactions. Issues of social inequality, citizen empowerment, democracy and education…
were approached by a quite a large section of academic works and institutional structures also as a major measurement challenge4. Academically, a real movement can be identified around the issues of
‘social indicators’ in a number of countries, and notably in the USA and France. In terms of influence on policy-making and as decision-aiding instruments, these indicator developments had however low influence, which made not only the public authorities’ fervour of investment tend to nil after a while, but was generating an increasingly difficult situation for academics to justify research in the field. One of the main reasons for this loss in interest in the configuration and measurement of social indicators appears to be the lack of a common understanding between experts (comprising policy actors) on how to assess social development in terms of methods and approaches (Boulanger, 2004; Perret, 2001; Cobb et al. 1998).
Without direct filiations, their work incidentally created the basis in the beginning 1990ies for the development of a now quite important social development indicator, the Human Development Index (UNDP, yearly). Lately, research activities in Europe are growing fast again into the development of
3 See for instance the developments in the field of Material Flow Accounting, but also the European NAMEA and SERIEE approaches.
4 For a passionate insight into the social indicators’ movement see Cobb and Rixford (1998) for the USA episode or Boulanger (2004) for a general overview. For an analysis of the rise and fall of ‘social indicators’ as academic branch, see Miles, 1985.
European social indicator reports (e.g. Atkinson et al., 2002) and even indices (e.g. Defeyt and Boulanger, 2003). The latter approaches are developing on a particularly active research community5. An increasing number of challenging new approaches to social indicators and indexes and reporting developed from the mid-nineties and are sustained by an increasing interest from national and international institutions.
After the Social indicators’ movement’s proponents scattered in the 1980ies, a strong latent expertise in multi-dimensional assessment through indicators subsisted, some of which found its way into ISD.
Not the least, the sociologists’ influence is echoed in the process-orientation of ISD-development, or more generally in the procedural and discursive understanding of the role of ISD in a context of
‘public choice’.
Environmental Policy Performance indicators
After environmental issues started to be institutionalized in the developed countries during the 1970ies and 1980ies, calls for the organization of a sufficient knowledge base for policy-making contributed largely to the development of environmental policy-making indicators. Their systematic and structured publications were - among other influences - forged by OECD’s ‘Environmental performance reviews’, which have been promoted by some of its member states since 1989 (see Boisvert et al. 1998). A demand which formalized (Lehtonen, 2003) in the beginning 1990ies into extensive inter-agency review processes of national performances in environmental policy-making.
Some of OECD’s early methodological and procedural contributions to environmental indicators - such as the P-S-R framework (see section 1.2.3.2) – contributed to characterize the logic of ISD- reporting.
Simultaneously, environment agencies at national and European level developed since the beginning 1990ies periodic State-of-the-Environment (SoE) reports. Largely based on data accounting for the biophysical and chemical quality of the environment, SoE evolved into reports based on more extensive usage of indicators and indicator frameworks (such as the P-S-R framework), while expanding their topics to the economic and social pressures and responses to environmental evolution.
These 3 episodes, from the SNA to the environmental performance indicators, contributed to characterize ISD in the mid-90’s as integrated, process-oriented, multidimensional, structured, systematic data-processing initiatives. From the sum of these characteristics, we will distil hereafter our own working definition for ISD. It should be understood from the history of ISD that the issues raised in this work have mostly been raised elsewhere or in earlier policy domains, be it in relation to
5 Gadrey and Jany-Catrice (2003) develop an exhaustive overview of the most interesting initiatives on social indicators (including indicators, indexes and reporting). Otherwise for an insight into the scientific production of the community, see journals such as “Social Indicators Research” (Kluwer).
economic accounting or social indicators or environmental reporting. Performing a more thorough analysis of earlier indicator movements would however exceed the scope of this work.
1.2.2 Definitions
In relation to our very specific context of ISD, it is OECD which provide us with the most commonly accepted definition6 of an indicator as “a parameter, or a value derived from parameters, which points to, provides information about, describes the state of a phenomenon/environment/area, with a significance extending beyond that directly associated with a parameter value” (OECD 1993, 2002, 2003). It should be emphasised at this stage that no direct reference is made to the relationship between indicators and ‘reality’ or ‘observation’, a link which obviously is not central to an indicator’s characterization. Indicators are meant to ‘point’, ‘provide’ and ‘describe’.
Slightly more subtle and elegantly, Boulanger (2004 : 3) defines an indicator as “an observable variable used to account for an unobservable reality”. And Boulanger to add a general definition of social indicators given by Bauer et al. (1966 : 1 in Boulanger 2004 : 3): “statistics, statistical series, and all other forms of evidence that enable us to assess where we stand and are going with respect to our values and goals”. 30 years after Bauer (et al. 1966), OECD explicitly acknowledged that the value-added provided by indicators is more than pure numbers and hence that values and personal (or community) objectives play an important role in constructing and using indicators.
Many authors inspired themselves from the OECD definition and from the working parties initiated at OECD level. Adriaanse (1993) developed in the context of environmental policy performance reviews for the Netherlands a widely used definition which appears to have direct filiations to OECD: "an indicator is supposed to make a certain phenomenon perceptible that is not - or at least not immediately - detectable. This means that an indicator has a significance extending beyond that [which] is directly obtained from observation. (...) Indicators generally simplify in order to make complex phenomena quantifiable in such a manner that communication is either enabled or promoted". Adriaanse inserted an argument which calls to sustain further procedural interest into indicators: they are meant to trigger communication among actors.
Building on Adriaanse’s procedural understanding, Rotmans (et al. 1997), quoted by his research associates (Greeuw et al. 2001), developed Adriaanse’s definition into: “Indicators describe complex phenomena in a (quasi-) quantitative way by simplifying them in such a way that communication is possible with specific user groups.” And to add that “the term ‘quasi’ indicates that, although indicators are mostly quantitative in nature, in principle they can also be qualitative. Qualitative indicators may be preferable to quantitative indicators where the underlying quantitative information is not available, or the subject of interest is not inherently quantifiable.”
Interestingly, if we step outside of the SD- and environmental indicator sphere, for instance by
6 We give here the current version of OECD’s definition of an indicator. Through the years, the wording of that definition was slightly adapted to policy-discourse. Essentially however, the definition remained constant over the last decade.
simply having a look at other OECD departments, the emphasis on what provides identity to an indicator slightly shifts. As one example among many possible, the OECD’s glossary on (development) evaluation, which was assembled in 2002, defines an indicator as a “quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor”. This definition is thus much more focused on policy performance: indicators are assessment tools which relate directly to policy. This understanding of the links between policy evaluation and indicators has found adherence also within the small emerging community of researchers active in the field of evaluation for sustainable development. The EASY-ECO7 (2002) research network issued a working definition for indicators, which they acknowledge as “a signal that reveals progress (or lack thereof) towards objectives; means of measuring what actually happens against what has been planned in terms of quantity, quality and timeliness”.
If, instead of leaving the environmental field, we turn to environmental indicators and have a look at ecologists’ definitions of indicators, we find a different emphasis. Bossel, an eminent (and emeritus) ecological systemist from Kassel University (Germany), elaborated a systemic view on sustainability (Bossel 1999) and its measurement. Consequently, for him indicators should be "system variables that provide us with all essential information about the health (viability) of a system and its rate of change, and about how that contributes to the goals we want to achieve with the help of that system"
(Bossel 1998 : 72).
Many more indicator definitions exist. Based on a generalized nuanced understanding of ISD, we use a working definition for indicators which is based on an earlier8 definition we contributed to:
Indicators for sustainable development provide an interpretation of the evolutions of stocks and/or flows in order to account for the human-environment interactions. Simplifying the complexity of reality, indicators are meant to participate to the self-generation of sustainable development by enhancing communication. Defined by technical, methodological and scientific conventions, the definition, selection and interpretation of indicators imply an articulation of scientific and societal values at various levels and depths.
In short, the following elements can be raised.
‘Simplifying the complexity of reality’ by using ‘Interpretation’
The indirect link between indicators (for sustainable development) and ‘reality’ should not be understood as a weakness of ISD, just as the inherent complexity in SD should not be regarded as undermining the viability of the concept or its assessment. The representativeness9 of indicators is not only influenced by the extent of the ‘measurable’ (e.g. advances in scientific knowledge and
7 Evaluation for Sustainability research network: http://www.sustainability.at/easy.
8 “An indicator is a sign or signal used to represent phenomena, events or complex systems. Always defined by conventions and values, an indicator renders an empirical interpretation of reality” (Zaccaï, Bauler, forthcoming).
9 Representativeness refers here to the capacity of an indicator to render phenomena with sufficient accuracy.
data-quality) and the ‘measured’ (e.g. advances in data-availability), but more so by the views of the indicator’s developers and users on the issues under assessment. For instance, the measurement of economic welfare is influenced by our understanding of the dynamic components of welfare (i.e. the measurable), by institutional arrangements between providers of data (i.e. the measured), and by the societal interpretation on what constitutes welfare. Obviously, the last factor influences largely the first two. In turn, the ‘societal’ translation of reality into indicators will be largely determined by the value referents of the implied actors.
‘Human – environment interactions’ considered as ‘stocks and flows’
Many different approaches to defining SD exist and in the second chapter we discuss some of them.
For the time being, we consider SD as the attempt to manage the interactions of human activities with their biological, chemical and physical environments. This implies an onion-like representation of SD: 1) at the heart of SD, we integrate the social, cultural, institutional, economic… dimensions, into what could be called the human dimension; 2) The environmental dimension encloses hierarchically the human dimension: without environmental sphere, no human activities. However this conception does not imply that we disregard the interactions within both spheres (for instance, between economic and social development), which are ruled by their own principles (e.g. for the human dimension: equity, justice…, or for the environmental dimension: thermodynamics, genetics…). In other words, our representation of SD is not environmentally centered, but recognizes that human activities are embedded in the environment and shouldn’t be conducted without due consideration of their consequences. The interactions within and between the dimensions are comprehended hereafter as variations in stocks and flows.
‘Self-generation of SD’ needs to ‘Enhance communication’
The initial political documents related to SD, such as Agenda 21, interpreted the comprehension of SD in terms of a number of conditions to be fulfilled. Some of the more fundamental conditions relate to issues of communication, such as transparency and openness of decision-making processes, stakeholder participation… Among others, indicators are seen as foundational instruments to simplify communication of facts and thus to participate to the achievement of SD via awareness-raising and/or capacity-building. Many authors nuance such an automatist vision of a self-generating SD and the prominent role given to ISD (and of information in general) in that dynamic. Fundamentally however, the prudent reading we apply here still acknowledges that configuring policy, i.e. inducing change, is very much linked to the development and communication of knowledge.
‘Conventions’ and ‘Values’
The always difficult link between ‘indicators’ and ‘reality’ opens large room for interpretation (and thus for dialogue) to those who construct, select and use indicators. Value judgments are inextricably linked to ISD at many levels from the conceptualization to the utilization of indicators. For instance, Cobb and Rixford (1998 : 1-3) wrote that if “technically speaking, an indicator refers to a set of statistics that can serve as a proxy or metaphor for phenomena that are not directly measurable”, it appears also that “the absence of objectivity stems primarily from inevitable biases in the selection of topics on which to gather data. There are also hidden biases in techniques of gathering and publication of data. The pretense of objectivity stands in the way of public appreciation of those biases”. The ‘intelligent’ management of these ‘value dialogues’ is one major condition for the successful translation of indicators into full-sized decision-aiding instruments.
Apart from value interferences at a methodological level, there exist many levels where judgments interfere with the definition of ISD. For instance, Boisvert (1998 : 107) refers to ‘purpose’ and
‘uncertainty’ in order to introduce two levels of judgment: “(the) main aim of indicators is to translate scientific information, full of uncertainty and inaccessible to laypeople, into operational data”. Indicators, as tools to enhance “bridging the gap”, are becoming a matter of translation of
‘Science’ to ‘Society’.
1.2.3 Types and typologies of indicators
Indicators are heterogeneous objects. Besides from introducing the diversity of possible approaches to indicators, we take the opportunity here to specify terminology10.
What is commonly called ISD refers to a series of different types of indicators with various levels of aggregation. Fundamentally, three types of aggregation are applicable to indicators:
Spatial aggregation: aggregating data and measurements originating from different monitoring spots to a degree which confers some spatial sense. Spatial sense can be obtained by using for instance institutional boundaries (e.g. national borders) as spatial aggregation masks, or by using natural or geographical referents (e.g. continents or river basins).
Temporal aggregation: aggregating data and measurements originating from different monitoring spots at different periods to a degree which confers some temporal sense.
Obviously only very few natural or social phenomena can be monitored continuously in order to be simply averaged. The rest of the measurements stems from data which have been aggregated from corrected samples taken at specific moments in time. In the context of SD, temporal aggregation can play a further role, namely to allow bridging the gap between current and past observations on the one hand, and future generations on the other.
10 More than before this part is resolutely relying on personal points of view. What is considered as an indicator and what as an indices is mostly a matter of personal choice more than of hard theory, especially in such a multi-
disciplinary field of enquiry than ours. The following hierarchical leveling of indicators should thus be taken as a way to clarify unambiguously the terms and wording used in the rest of the text.
