The utility of data –
from a user perspective
Professor Denise Lievesley
Head of School of Social Science and Public Policy, King’s College London and
Chair, European Statistical Advisory Committee
and
Chair, Methods Committee European
Themes
{ To introduce the role of ESAC
{ To discuss quality from a user perspective
The European
Statistical Advisory
Committee
The ESAC - History
Need for profound reform of role, mandate and composition of CEIES
November 2006 Commission proposal
11 March 2008, Adoption of Decision No 234/2008/EC on the ESAC
Inaugural meeting on June 2009
The ESAC – Mandate
New Committee shall assist the European Parliament / Council / Commission in
ensuring that user requirements and the costs borne by information providers and producers are taken into account in
coordinating the strategic objectives and priorities of the Community’s statistical
information policy
The ESAC – Tasks
OPINION
on Community statistical programme, in particular on its
relevance to requirements of European integration
relevance to Community activities
balance as regards priorities and resources and possibility to re-prioritise statistical work
adequacy of resources for its implementation and appropriateness to users’ needs
costs and possibilities of reducing response burden
own-initiative opinions/reports on user
requirements and costs borne by data providers
The ESAC – Tasks (continued) ADVICE
Point out necessary new statistical activities
Advise the Commission how to improve the relevance of Community statistics to users
Relations with Community institutions/other bodies At request of the European Parliament/ Council/
Commission, ESAC shall deliver an opinion
relating to user requirements and costs incurred by data suppliers in
development of the Community’s statistical information policy
priorities of the Community statistical programme
evaluation of existing statistics
data quality and
dissemination policy
The ESAC – TOOLS
Plenary meetings
Establishment of temporary working parties
Commission of studies
Organisation of seminars For more information:
http://epp.eurostat.ec.europa.eu/portal /page/portal/esac/introduction
The ESAC – Composition
24 members, appointed for 5 years, renewable once:
12 appointed by Commission
• users
• respondents
• other stakeholders
11 directly appointed by
• European Parliament
• Council
• Eur. Economic and Social Committee
• Committee of the Regions
• European Central Bank
• ESSC (2 members)
• Businesseurope
• ETUC
• UEAPME
• Eur. Data Protection Supervisor
Director-General of ESTAT
challenges
{ Different mandates of group
{ Emphasis on response burden
{ Range of users
{ How to connect with users
{ Coverage of countries
{ Resources, resources, resources
{ Deciding where we add value
Adding value
{ Principle of subsidiarity (consultation on national statistics better undertaken
nationally – but …
{ Sharing good practice across countries
{ Understanding needs for European data
{ Influencing European practice
The ESAC Æ state of play
Set-up of 3 Temporary Working Parties on
“GDP and beyond and Stiglitz-Sen report”,
“Access to confidential data for scientific purposes”
“Quality”
European Statistics Code of Practice 2005
PRINCIPLE 11: RELEVANCE
European statistics must meet the needs of users.
- Processes are in place to consult users, monitor the relevance and practical utility of existing statistics in meeting their needs, and advise on their emerging needs and
priorities.
– Priority needs are being met and reflected in the work programme.
– User satisfaction surveys are undertaken periodically.
Achieving relevance
(to Government)whilst ensuring independence
(fromGovernment)
“The Marriage of Impartiality and Practical Utility”
Paper given by Richard Alldritt and myself at the ISI in Durban, 2009
UN Fundamental Principles of Official Statistics
Principle 1
“Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public ... To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies..”
Words: relevance or utility?
{ “Relevance” more often used than
“utility”, but relevance to what? The wishes of government?
{ Better to focus on utility or fitness for purpose
Quality of data
Currency punctualityand
Currency punctualityand Relevance
to policy Relevance
to policy
Potential for disaggregationPotential for disaggregation
Coherence Coherence Consistency
over time and space Consistency over time and space
Validity reliabilityand
Validity reliabilityand
Comparability through standards Comparability
through standards
Accessibility and affordability
Accessibility and affordability
Efficient use of resources Efficient use of resources
Impartiality
{ The role of statisticians:
z to inform political debate and decisions without taking part
z to promote the collection and use of statistics for policy purposes
{ Fear exists that enhancing statistical utility will compromise impartiality
{ Not impartial Ö lack of trust Ö low utility
Are we too shy?
{ Are statistical offices so afraid of being accused of political motives that they dare not make reports useful?
{ Is the only answer to avoid
commenting on the relevance of the statistics to public debate?
“Trust comes on foot, but leaves on horseback.”
Dutch statesman, Johan Thorbecke
The policy making process
{ Policy making is the process by which
governments translate their political vision into programmes and actions to deliver desired
changes in the real world
Ideology
Lobbies
Values/
beliefs
Tradition
Self interest
Judgement
Campaign promises
Expert Acceptability
All evidence is imperfect
“The absence of excellent evidence does
not make evidence-based decision making impossible: what is required is the best
evidence available not the best evidence possible”
Muir Gray 1997
“ Evidence rarely provides neat and tidy prescriptions to decision makers as to
what they should do. Often it generates more questions to be resolved ”
Evidence is sometimes resisted...
“
There is nothing a
government hates more
than to be well-informed:
for it makes the process of arriving at decisions much more complicated and
difficult
. ”John Maynard Keynes
Inconvenient truths
{ Governments prefer good news stories
{ Bad news stories may be delayed or buried
{ They are often too focussed on populism
{ The government’s horizons can be
shorter than those of social researchers!
{ They prefer their own spin to the
interpretation of the statistician/social
Sir Gus O’Donnell (UK Cabinet
Secretary)
{ “I want [the ONS] to be boring, to put out the plain facts, and nothing but the facts, and on clear, predictable deadlines,” he said. It would then be for politicians and government press officers to interpret the figures, he added.
Response of the Royal Statistical Society
{ It is clearly the task of official statisticians to interpret the figures in a statistical context, to facilitate understanding and avoid
misunderstanding.
{ The Code of Practice of the UK Statistics Authority explicitly states that Official
statistics, accompanied by full and frank commentary, should be readily accessible to all users and that all UK bodies that are responsible for official statistics should
prepare and disseminate commentary and analysis that aid interpretation, and provide
Important aspects of building trust
{ Autonomy of statistics office
{ Statistical legislation
{ Existence of an independent statistical board
{ Development of codes of conduct
{ Breaches of the code identified, investigated and publicised
{ Appointment of head of statistics should be removed from the political process
{ Statistical processes should be externally audited
{ Audit body should report to Parliament
Important aspects of building
trust – the statistical environment
{ Media educated on the value and quality of statistics
{ The public are statistically literate
{ Data are made available for open and transparent debate
{ Users are involved in setting the agenda for statistics – and are seen to be
Concerned about an over-
emphasis on response burden?
{ Suppliers who understand the value of data and who use it do not see it as a burden
{ The value of statistics to society must not just be asserted; it must be
demonstrated
{ But we understand the reality of the funding environment
Demonstrating the value of statistics to society
“Were a balance sheet for official statistics to be prepared, the costs would be clear enough. The benefit, or value, would however be found to be much more diffuse and harder to treat in traditional accounting terms. Given this, it is possible that the vital asset that official statistics represent is
undervalued in public sector planning processes.
And we observe that little systematic consideration is given to how the public value could be
maximised”.
(UK Statistics Commission, The Use Made of Official
Interesting conundrum
Some of the national statistics offices judged to be the ‘best’ by fellow
official statisticians are not viewed this way by users
Why?
{ Often the assertion of independence is viewed as arrogance
{ Official statisticians should balance
professional confidence with humility, a willingness to learn from users and the
ability to put oneself in the users’
position.
{ Determining fitness for purpose requires
Challenges
{ Lack of mechanisms to consult users
{ Fragmentation of data producers
{ Poor understanding of the cost to a country of not using data
{ Increasing use of administrative data
Value of relationships with
{ Data intermediaries
{ Expert users
{ The professional statistical community
Data intermediaries
{ Data archivists, librarians
z Organise user groups
z Capture information on data usage
z Expertise in data supply
z Metadata standards
z Can help to produce quality documentation
z Ensure that the documentation incorporates user expertise
Foster engagement with expert users
{ Even more important as there is an increasing pressure on statistical
resources
z Prioritisation must involve discussion with users
z Expert users can provide methodological expertise
z Satisfied users help to fight for resources for statistics
Professional community of statisticians
{ Training and mentoring
{ Accreditation and support
{ Professional ethics/ codes of conduct
{ Training on ethics
{ Environment in which people can discuss problems
{ Support for statisticians in difficult situations
