Expert-based ecosystem services capacity matrices: Dealing with scoring variability

(1)

HAL Id: hal-01681625

https://hal.archives-ouvertes.fr/hal-01681625

Submitted on 11 May 2018

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Expert-based ecosystem services capacity matrices:

Dealing with scoring variability

Carole Sylvie Campagne, Philip Roche, Frédéric Gosselin, Leita Tschanz,

Thierry Tatoni

To cite this version:

Carole Sylvie Campagne, Philip Roche, Frédéric Gosselin, Leita Tschanz, Thierry Tatoni.

Expert-based ecosystem services capacity matrices: Dealing with scoring variability. Ecological Indicators,

Elsevier, 2017, 79, pp.63-72. �10.1016/j.ecolind.2017.03.043�. �hal-01681625�

(2)

EcologicalIndicators79(2017)63–72

ContentslistsavailableatScienceDirect

Ecological

Indicators

jou rn al h om ep a g e :w w w . e l s e v i e r . c o m / l o c a t e / e c o l i n d

Original

Articles

Expert-based

ecosystem

services

capacity

matrices:

Dealing

with

scoring

variability

Carole

Sylvie

Campagne

a,b,∗

_,

_Philip

_Roche

a

_,

_Frédéric

_Gosselin

c

_,

_Léïta

_Tschanz

b

_,

Thierry

Tatoni

b

a_Irstea,_UR_RECOVER,₃₂₇₅_route_de_Cézanne,_CS_40061,₁₃₁₈₂_{Aix-en-Provence}_Cedex_5,_France

b_Institut_{Méditerranéen}_de_{Biodiversité}_et_d’Ecologie_marine_et_continentale_(IMBE),_UMR_7263,_Aix_Marseille_Université,_CNRS,_IRD,_Avignon_Université,

FSTSt-Jérôme,13397Marseille,France

c_Irstea,_UR_EFNO,_Domaine_des_Barres,₄₅₂₉₀_{Nogent-Sur-Vernisson,}_France

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received31January2017

Receivedinrevisedform10March2017 Accepted23March2017

Availableonline15April2017 Keywords:

Ecosystemservices Capacitymatrix Conﬁdenceindex Uncertainty Expertpanelsize Participatoryassessment

a

b

s

t

r

a

c

t

Capacitymatricesarewidelyusedforassessmentofecosystemsservices,especiallywhenbasedon participatoryapproaches.Acapacitymatrixisbasicallyalook-uptablethatlinkslandcovertypesto ecosystemservicespotentiallyprovided.ThemethodintroducedbyBurkhardetal.in2009hassince beendevelopedandappliedinanarrayofcasestudies.Hereweadresssomeofthecriticismsontheuse ofcapacitymatricessuchasexpertpanelsize,expertconﬁdence,andscoringvariability.

Basedonthreecase-studycapacitymatricesderivedfromexpertparticipatoryscoring,weusedthree differentapproachestoestimatethescoremeansandstandarderrors:usualstatistics,bootstrapping, andBayesianmodels.Basedonaresamplingofthethreecapacitymatrices,weshowthatcentralscore stabilizesveryquicklybutthatintersamplevariabilityshrinksafter10–15expertswhilestandarderror ofthescorescontinuestodecreaseassamplesizeincreases.Comparedtousualstatistics, bootstrap-pingmethodsonlyreducetheestimatedstandarderrorsforsmallsamples.Theuseofconfidencescores expressedbyexpertsandassociatedwiththeirscoresonecosystemservicesdoesnotchangethemean scoresbutslightlyincreasesthestandarderrorsassociatedwiththescoresonecosystemservices.Here, computationsconsideringtheconfidencescoresmarginallychangedthefinalscores.Nevertheless,many participantsfeltitimportanttohaveaconfidencescoreinthecapacitymatrixtoletthemexpress uncertaintiesontheirownknowledge.Thismeansthatconfidencescorescouldbeconsideredas sup-plementarymaterialsinaparticipatoryapproachbutshouldnotnecessarilybeusedtocomputefinal scores.

Wecomparedusualstatistics,bootstrappingandBayesianmodelstoestimatecentralscoresand stan-darderrorsforacapacitymatrixbasedonapanelof30experts,andfoundthatthethreemethodsgive verysimilarresults.Thiswasinterpretedasaconsequenceofhavingapanelsizethatcountedtwice theminimalnumberofexpertsneeded.Bayesianmodelsprovidedtheloweststandarderrors,whereas bootrappingwithconﬁdencescoresprovidedthelargeststandarderrors.

Theseconclusionspromptustoadvocatewhenthepanelsizeissmall(lessthan10experts),touse bootstrappingtoestimateﬁnalscoresandtheirvariability.Ifmorethan15expertsareinvolved,theusual statisticsareappropriate.Bayesianmodelsaremorecomplextoimplementbutcanalsoprovidemore informativeoutputstohelpanalyzeresults.

Abbreviations: ES,ecosystemservice;ET,ecosystemtypes;SD,standard devi-ation;SE,standarderrors;CP,cutpoint;RNP,RegionalNaturalPark;RNPBP,“Les BaronniesProvenc¸ales”RNP;RNP-SE,“Scarpe-Escaut”RNP;RNP-SEwet, “Scarpe-EscautRNPwetlandsonly”.

∗ Correspondingauthorat:Irstea,URRECOVER,3275routedeCézanne,CS 40061,13182Aix-en-ProvenceCedex5,France.

E-mailaddresses:sylvie.campagne@irstea.fr,sylviecampagne@gmail.com

(C.S.Campagne),philip.roche@irstea.fr(P.Roche),frederic.gosselin@irstea.fr

(F.Gosselin),l.tschanz@imbe.fr(L.Tschanz),t.tatoni@imbe.fr(T.Tatoni).

1. Introduction

Ecosystemservice(ES)isapopularandwidelyrecognised con-cept (Burkhard et al., 2012; Fisher et al., 2009), and the term ‘ecosystemservices’hastranslatedfromscientiﬁcstudiesintothe mainstreamvocabularyofstakeholdersandexperts(Jacobsetal., 2014).IncreasingdemandfrompolicymakersliketheEuropean Commission has prompted the development of an array of ES

http://dx.doi.org/10.1016/j.ecolind.2017.03.043

(3)

assessmentandmappingmethods(Martnez-HarmsandBalvanera, 2012;Willemenetal.,2015).

Onesuchmethodthatisgaininggroundisthe“capacitymatrix approach”,whichwaseventoutedas“themostpopularES assess-menttechniquetoday”(Jacobsetal.,2014).Thecapacitymatrixis basicallyalook-uptablethatlinkslandcovertypestoecosystem servicespotentiallyprovided(Burkhardetal.,2009).

Sincethe“matrix”ﬁrstintroducedbyBurkhardetal.in2009, themethodhasbeendeveloped andappliedinanarrayofcase studies(e.g.Hermannetal.,2013;Krolletal.,2012;Stolletal., 2014;Vihervaaraetal.,2010;etc.).Basedonexperts’knowledge, itgivesaquickassessmentofESpotentiallyprovidedinanarea (Vihervaaraetal.,2012;Stolletal.,2014).TheESconceptmakes thematrixmethodmobilizetheESconceptinwaythatmakesit easyforstakeholderstounderstandandappropriate.Itisa peda-gogicaltoolthathasprovenitsutilitybytargetingprioritiesand highlightingmanagementhotspots.Theapproachcanbeapplied atdifferentscales(e.g.Stolletal.,2014orHermannetal.,2013). Alackofquantitativedataandtheirspatialheterogeneityraises issuesthatcanbebypassedbyaskingexpertstoestimatescores. Dependingontheconcertationprocessapplied,datacanbe devel-opedbyconsensusamongthedifferentexpertsofaterritory.The methodisalsoﬂexibleenoughtointegrateallkindsofdata—from modelsormeasurementsalike(Burkhardetal.,2014).

Someresearchershavestartedtostudythelimitsofthismethod, likeJacobset al.(2014) whopoint out itspoormethodological transparency,lackofreproducibilityandlackofappropriate factor-ingonuncertainty.Houetal.(2012)alsodiscusstheuncertainties relatedtothematrixmethod.Theuncertaintyofexperts judge-mentsisoftencitedasalimit,butfewhaveanalyzeditorintegrated itinESstudies(Seppeltetal.,2011;Houetal.,2012;Vihervaara etal.,2012).Basedonthecombinationofexperts’judgments,the scoringinthecapacitymatrixmaycarrytwosourcesof uncertain-ties:

–Variabilityamongexperts:variabilityoftheexpertisewithinthe chosenexpertsandofmoregeneralknowledge(professionalor personalknowledgedependingontheirexperiences)(Houetal., 2012).

–Variabilityofeachexpert:theconﬁdencetheexperthasinhis/her ownscores(Jacobsetal.,2014).

Theobjectiveofourstudyistwofold.First,weaimtointegrate thedifferentvariabilitiesintheﬁnalscoreofthecapacitymatrix andpresentandcomparedifferentapproachestocomputingthem. Second,weaimtoidentifyaminimalsizeoftheexpertpoolneeded toobtainareliableestimateofthemeanofthescoresandasmall SEofthismean.

Inorder tomeet theseobjectives,we beginby definingour capacitymatrices,theexperts,andhowthescoringwasdone.The confidencescoreweaddedonthecapacitymatrixisalsodetailled. Inthesecondsection,wepresentthreesetsofapproachestothe finalscoresonamatrix:rawparametricapproaches(meanand weightedmean),bootstrapmodels,andBayesianmodels.Foreach approach,wehavetwocalculations:meansofscoresthatexperts expressed,andmeansintegratingametricoftheexpert’s confi-denceonhis/herownscores.Thefinalscoresinthefinalmatrix thatincorporatesallexpertsscoresisthusestimatedwith6 calcu-lations.Mostexistingcapacitymatrices(e.g.Stolletal.,2014)using expertknowledgeareexpressedasmeanofscoresoftheexpert panel,sowestartbypresentingtherawparametricapproach.The bootstrapmodelenablestoestimatedifferentstatisticsby assum-inganindependentsamplingfromanunknowndistributionand tointegrateuncertainties.TheBayesianmethodsthatweusedare elaborateparametricstatisticalmodelsthatenabletointegrateand estimatethedifferentkindsofuncertaintiesinthestatistical

anal-ysis.Werestrictourselvestothesethreesetsofstatisticalmethods. Thisworkisthefirstcomparisonofthreecalculationsappliedon capacitymatrixscores.Wearenotsettingouttoidentifythebest calculationoffinalscorebuttohighlightthevariouspossibilities andtheirrelatedadvantagesanddisadvantages.Inthethird sec-tion,wepresenttheresultsofthethreecalcultationsononematrix andlookatthefinalscoresandtheirvariabilitiesonthreecapacity matriceswithagrowingnumberofexperts.Finaly,weconclude withrecommandationsonusingthecapacitymatrixapproach.

ThedatausedinthispapercamefromthreeESassessments: ESprovided by land-covertypesin the ‘BaronniesProvenc¸ales’ Regional Natural Park (RNP) (associated scores noted RNP-BP) andtwo ESassessmentsinthe‘Scarpe-Escaut’ RNPinnorthern France—oneonESprovidedbywetlands(associatedscoresnoted RNP-SEwet)andanotheronESprovidedbyallland-covertypes (associatedscoresnotedRNP-SEall).

2. Data

2.1. Studysites

The Baronnies Provenc¸ales RNP ( http://www.baronnies-provencales.fr/) is a sub-mountainous rural area in Southern FrancelocatedatthecrossroadbetweentheAlpsand Provence inﬂuences.Createdin2015,itisthelatestRNPinFrance,taking the total to51. The capacity matrix made in 2014 was based on the Park project of 2350km2 _and ₁₃₀ _{municipalities.} _This nature-preserve territoryis recognisednationallyfor itsunique landscapes, rich “terroir”, built heritage (terraces in dry stone, hilltop villages)and agriculture (orchards,olive groves, linden, lavender,thyme,rosemary,andmore),aswellasitsremarkable geologyandbiodiversity.

The Scarpe-Escaut RNP (http://www.pnr-scarpe-escaut.fr/en) innorthernFrance,neartheBelgianborder,extendsover430km2 crossedbytheScarpeandtheEscautriverswith55municipalities. Itistheoldestofthe51FrenchRNP.Itisalsothelargest Euro-peanpark,astogetherwithitsBelgianneighbour,thePlainesde l’EscautNaturalPark,theyformtheHainautcross-borderNature Park.TheScarpe-EscautRNPisespeciallymarkedbythewet low-landplainaroundtheScarpeandtheEscautrivers.Asaperi-urban area,urbanpressureishigh(useofspace)inalandscapeformed byamosaicofagriculturalandnaturalenvironments(crops, grass-lands,woodlands,marshes,ponds...)andurbanizedareas.Water is everywhere, and man hasbeenmanaging it for centuriesto developkeyactivities(drainage,landuse,channelingofrivers...). Fordecades,wetlandshavebeenconsideredalessattractive land-scape,andwetmeadowshavebeendecliningunderurbanpressure or exploitedas proﬁtablesites for agriculturalproduction such aslivestockandaslandscapedrecreationalponds.Perceptionsof wetlands today are either badfor certain localstakeholders or nonexistentfor thewidercommunity, despitetheirimportance asESprovidedtotheterritory.Thisperceptiondeﬁcitprompted astudyoftheESprovidedbythewetlandtypesin2015.Afterthe positivelocalfeebackontheinitiative,themethodwasappliedin 2016toalllandcovertypes.

2.2. Data

2.2.1. Thecapacitymatrices

We defineES as goods or services provided by ecosystems thatdirectlyorindirectlybenefithumans(MillenniumEcosystem Assessment,2005).For provisioningservicesandregulating ser-vices,theESlisthasbeenbasedontheEuropeanCICESclassification (Haines-YoungandPotschin,2013).Weconsideredprovisioning services,regulatingservicesandculturalservices.

(4)

C.S.Campagneetal./EcologicalIndicators79(2017)63–72 65

BesidesthescorevalueforeachES/ecosystemtype combina-tion,theexpertswereaskedtoprovideanindexofconfidencein theirscoreforeachESandeachecosystemtypeforthelasttwo datasets.Thisconfidenceindexisusedtoestimatetheimpactof expertconfidenceonthecapacityscoreandtheirvariations.

AsthispaperfocusesnotoncapacityscorestoestimateES capac-ityperse butonthemethodologyusedtoobtainexpert-based capacitymatricesandevaluatescoreconﬁdencelevels,detailedES andecosystemtypesforthedatasetsaregivenassupplementary materials.Ourdatasetiscomposedofthreesetsofexpertscoring exercicesusedtoproducethreedifferentcapacitymatrices.

Theﬁrstsetwascompletedin2014andconsistsofapanelof 23expertsintheRNP-BP.Thiscapacitymatrixcounts33 ecosys-temtypes(ET)(15agriculturehabitats,12foresthabitatsand6 aquaticandurbanhabitats)relatedtolandcovertypesusedinthe regionallandcovermapderivedfromtheNationalForestInventory (IFNv2),theagriculturaltypologyoftheMinistryofAgriculture, FoodandForestry,andtheAgencyforServicesandPayment(ASP; 2010), calledtheGraphicParcelRegister(2011), aswellasthe CRIGE-PACAregionallandcoverclassiﬁcationcalledOcsol(2006)(a regionalimplementationofCorineLandCoverwithahigherspatial resolutionof1/50,000)by22ES(8provisioningservices,9 regu-latingservicesand5culturalservices),giving726scores(Tschanz etal.,2015).Thistotaldatasetisthus726ES/ETexpertscores×23 experts.

Thesecondsetwascompletedin2015andconsisstofapanelof 30expertsintheScarpe-EscutRNPdealingexclusivelywith wet-land(RNP-SEwet).Inthistypology,ETisassociatedwithwetland environmentsbasedonalocalwetlandsmap.Weidentiﬁednine ET(2foresttypes,3agriculturaltypes,4semi-naturaltypes)and 17ES(5provisioningservices,7regulatingservicesand5cultural services),giving153scores.Thistotaldatasetis153ES/ETexpert scores×(9+17)conﬁdencescores×30experts.

Thethirdsetwascompletedin2016andconsistsofapanelof 17expertsintheScarpe-EscautRNPasawhole(notedRNP-SEall) intendedtobemoreexhaustiveasitintegratesalllandcovertypes. Thiscapacitymatrixcounts33ET(6aquatichabitats,13agriculture habitats,7foresthabitatsand7anthropizedhabitats, correspond-ingtothelandcovertypesintheARCHmap,www.archnature.fr) by25ES(9provisioningservices,11regulatingservicesand5 cul-turalservices),giving726ﬁnalscores.Thetotaldatasetis726ES/ET expertscores×(33+25)conﬁdencescores×17experts.

Thesesthreematriceswereproducedindifferentcontextsand withdifferentsobjectives.WhiletheRNP-SEwetmatrixwas com-pletedinacontextofexperts’concertationandoutreacheducation tolocalcitizens,theothertwowerecompletedforamore exhaus-tive spatial assessment, so we did not use identical ET or ES typologiesbutinsteadranlocalcontextualadaptationsbasedon thelandcovermapsavailableoneachapplicationsite.

2.2.2. Expert-basedparticipatorymethod

Thecapacitymatrixaimstoproduceestimatesofthe biophysi-calcapacityoftheecosystemtoprovideES.Werestrictedthepanel ofelligibleexpertstothosehavingbothlocalandglobal ecologi-calknowledgeinordertotakeintoaccountallmajorecosystem typesandallmajoractivitiesappliedonthem.Wegenerallytried tofollowtherecommendationsofJacobsetal.(2014)onforminga relevantsampleofexpertswithspecificaffinitytotheirterritory. Ourdefinitionof‘expert’wasapersonwithextensiveknowledge orskillsbasedonresearch,experience,oroccupationinaparticular field.

Theexpertsconsideredincludedresearcherswithexpertisein ecologyand/orES,projectorsitemanagers,techniciansworkingon environmentalorecologicalﬁelds,andheadsofterritorial organi-zationssuchasthewateragency,regionalchamberofagriculture, regionalprofessionalcentreforforestowners,regional

environ-mentalsciencecouncil, regionalconservatoriesofnaturalareas, nationalbotanicalconservatory,localorregionaldepartmentsof environmentalaffairs.Thedepartmentalfederationofhuntersand ﬁsherswasalsorepresented,alongwithassociationsfor environ-mentalprotectionornaturalists.

ForeachoftheESassessments,theparticipantswereinvitedto aworkshopdedicatedtofillingoutcapacitymatrixscores.Any par-ticipantwhowasunabletoattendtheworkshopwasinterviewed individually. During theworkshopand thepersonal interviews, participantswereinformedonthestate-of-the-artinES,thestudy, themethods,andthelistofESandET.Wetooktimetodiscussand clarifyallthedefinitionsinvolvedinthematrixandthescoring.We proposedtocompletethematrixbycolumnsandtogiveascoreby comparingthedifferentEScapacitiesofeachET.Ourexperience hasproventhatthiswasaneffectivewaytofillinthematrix.The workshopslastedoneday,withthemorningsessiondedicatedto presentingalldefinitionsandtheafternoonleftforparticipantsto fillinthematricesanddiscusstheirunderstandingofthescorings. Inbothsituations(workshoporpersonalinterviews),welet peo-plegivetheirownscoreindependently.Thedifferencebetweenthe twoapproachesliesinthedialogueinitiatedintheworkshopafter thescoring,whereeveryonewasgiventimetovoicetheirchosen scoreandopenadialogueonanydivergences.Atthebeginningof theworkshop,wedefinedandspecifiedtherulesofdialogue: free-domofspeechandtoholddivergentopinions,andthepossibility ofconstructivecriticism.

2.2.3. Scoring

ThecapacitiesofthedifferentETtoprovideanindividualES werequantiﬁedfollowingthescalegiveninBurkhardetal.(2009), i.e.“1=lowrelevantcapacity, 2=relevantcapacity,3=moderate relevantcapacity,4=highrelevantcapacity,5=veryhighrelevant capacity”and“0=norelevantcapacity”.FortheRNP-SEwetmatrix, “0=norelevantcapacity”wasnotconsideredsincealltheselected ESwereprovidedtosomelevelbyallthewetlandhabitats dis-cussed.

Intherestofthepaper,“expertscore”referstothepotentialES capacityvaluethatindividualexpertsprovidedandisnotedY_k,i,o, whereiistheES,kistheETandoistheexperts(thetotalofall expertsinvolvedisnotedn),and“ﬁnalscore”referstotheaverage potentialcapacityvaluecomputedfromasubsetorallexpertscores andisnotedk,i,o.

2.2.4. Theconﬁdencescore

Inanyapproachbasedonexperts’knowledge,oneofthe difficul-tiesistoassessthedegreeofconfidenceofeachindividualexpertin theirownscore.AsJacobsetal.(2014)stressed“confidencereporting isparamountforcommunicationofresultsandqualitycomparison”. Weproposetoquantifyascoreoftheconfidencetheexperthason theirscores,where“confidencereferstothedegreeofconfidencein beingcorrect”(Jacobsetal.,2014).Eachexpertwasaskedtostate theirconfidenceintheirknowledgeonETandESusingaconfidence scorerangingfrom:“1=Idon’tfeelcomfortableonmyscore,2=I feelfairlycomfortableonmyscoreand3=Ifeelcomfortableonmy score”.

Thus,foreach expert(notedo) oftheRNP-SEwetmatrix,we had17conﬁdencescoresforeachESnotedVES

i,oand9conﬁdence scoresforeachETnotedVET

k,o.WeconsideredtheseETandES con-fidencescoresasmarginsofatable,andcomputedaconfidence scoreforeachETandESbymultiplyingthem:Vk,i,o=Vk,o×Vi,o.The confidencescoreobtainedcouldtakevaluesof1,2,3,4,6or9that werecodedonascaleof1–6(bychangingtheoriginal“6”to“5” andtheoriginal“9”to“6”)tofaciliatetheanalysis.

ForthebootstrappingandtheBayesianapproachincorporating theconfidencescore,wefirstestimatedthevariationof ES×ET score associated witheach confidence scorevalue basedonan

(5)

additionaldirectsurveywith10expertsoftheRNP-SEwetmatrix (allocatedwithinthedifferent ecologicalspecificities/expertises andknowledge)in order toexpresstheuncertaintyunderlying theirconfidencescores(citedaslevelsofuncertaintyandnoted Uk,i,o).Throughconcreteexamples(combinationsofESandETwith differentconfidencevalues),weaskedthemtoexpressarangeof ES×ETscores(0;0.33;0.5;1or2upanddowntheirownscore) thattheycouldhaveconsideredonasetofspecificET×ES com-binationswithdifferentassociatedconfidencescores.Wetriedto coverasmanyoftheexperts’questionsofpossiblecombinations ofconfidencescoresVk,i,o.FromthisdatasetwecomputedtheSEof ET×ESscoresassociatedwitheachconfidencescore.

3. Statisticalanalysis

3.1. Usualstatistics

Mostcapacitymatricesusingexpertknowledgeareexpressed asmeanofthescorevaluesoftheexpertpanel.

First,aﬁnalcapacitymatrixcanbeobtainedwithusualdirect statisticsthroughthemean,alsocitedasaveragescore(noted␮ˆ1k,i) anditsSD(noted␴ˆ1k,i).

FromtheSD,theSEofthemeanscoreisestimatedbydividing by√ncalculatedas ˆse1k,iandasfollowsfortheSEsofthemean:

ˆ se1k,i =

ˆ _√1k,i

n

Wealsoused theconﬁdenceindexscore (1–6)asa weight-ingcoefﬁcientfortheusualstatistics.Theweightedmean(noted

ˆ

␮1k,i,o )canbecalculatedwithdifferentvaluesdependingonstudy, context,etc.Hereweusedconﬁdencescoreasweightingfactor fortheexpertscores,sotheweightedscoreistheexpert’sscore multipliedbyconﬁdencescore,asfollows:

Y_k,i,o =Yk,i,o×Vk,i,o

Accordingly,theweightedmeanofthescoresiscalculatedas follows: ˆ ␮1k,i = i,o_Y k,i,o i,o_V k,i,o

ThentheSDoftheweightedmeans(notedˆ3

k,i)iscalculatedas follows: ˆ 1 k,i=

i,o

_Y_k,i,o₋_␮3 i

2 ∗Vk,i,o

i,o_V_k,i,o

FromtheSDs,theSEsofthemeanofscoresisestimatedby dividingby√_nandthuscalculatedas ˆ_se1_k,i₌ˆ

1 k,i

√

n andasfollowsfor theSEsoftheweightedmean:

ˆ se1k,i = ˆ _√1k,i n 3.2. Bootstrappingstatistics

Assuminganindependentsamplingfromanunknown distribu-tion,thebootstrapmethodenablestoestimatedifferentstatistics inabetterway.TheMonte Carlosimulation approachprovides approximationsof different statistics by independently repeat-ingNtimesaresamplingwithreplacementfromagivensetofn

observations.Eachresamplingprovidesanindependentrealization fromtheinitialdatasetfromwhichNrealizationsofthestatistics areproduced,providingadistributionofthestatisticsconsidered. TheboostrappedMonteCarlosimulationsweredoneusingR3.3.1, whichfeaturesthe“boot”packagebutwepreferred,here,towrite ourownscriptforgreaterﬂexibility.

Inourcase,n,thenumberofobservations,isequaltothenumber ofexperts.Weestimatedtwostatistics:␮ˆ2

k,i themeanofscores and ˆse2k,i=ˆ

2 k,i

√

n theSEofthemeanofscores,withNthenumberof repetitionsbeingequalto500.Numberofrepetitionsisnotusually crucialonceitpasses100(Efron,1981).

Withthebootstrappingstatistics,thevariationinES×ETscore associated witheach conﬁdencevalue wasusedin a bootstrap MonteCarloapproachtoestimate␮ˆ2

k,ianditsSE(noted ˆse2

k,i) tak-ingintoaccounttheconﬁdencescoreassociatedtoYk,i.Weadded inthebootstrapscriptarandomerror_ε_k,i,odrawnfromanormal distributionoferrorsassociatedwitheachleveloftheconﬁdence scores,whichgivesεk,i,o ∈N

0,ˆk,i,o,c

withcbeingthevalueof theconﬁdencescore(0,...,6).

3.3. Bayesianapproach

For the Bayesian analyses, we used two versions of a het-eroscedastic ordered probit model (Litchﬁeld et al., 2012; see SupplementaryMaterials)forYk,i,o.Indeed,theprobabilities asso-ciated with each observation Yk,i,owere calculated as if the observations stemmedfroma latentrandom variableωk,i,o fol-lowinganormaldistributionaccordingtothefollowingformulas:

P(Yk,i,o=1)=P(ωk,i,o<CP1,i) P

Yk,i,o=2

=P

CP1,i<ωk,i,o<CP2,i

P

Yk,i,o=3

=P

CP2,i<ωk,i,o<CP3,i

P

Yk,i,o=4

=P

CP3,i<ωk,i,o<CP4,i

P

Yk,i,o=5

=P

ωk,i,o>CP4,i

wherethecut-pointsCPj,iwere(i)ﬁxedto0and1forCP1,iandCP4,i; and(ii)estimatedbetween0and1withinthemodelforCP2,iand CP3,i, via the following formulas: CP2,i=1/

1+exp

−CP02,i

and CP3,i=1/

1+exp

−CP02,i−exp

−CP03,i

with both CP02,i andCP03,i havingaGaussianprior distributionof

ℵ

(0,5). CPj,icut-pointswereconsideredtovaryfromoneEStoanother becauseweconjecturedthatthespanbetweenscorevalueswould varyfromone EStoanother.We checkedthevariationsofthe estimators of CPj,i withi and foundthat, for some ESat least, thecut-pointvalueswereverysigniﬁcantlydifferentfromother values.

The two parameters describing the distribution of ωk,i,o werethelocation or meanparameter ˆ3

k,i,oand the dispersion parameter ˆ_se3_k,i,o corresponding to the SEof the latent normal distribution. After comparing four models (see Supplementary Materials),weretainedtwodifferentmodelsthatcontained dif-ferentparametrizationsoftheheteroscedasticity(i.e.inparameter

ˆ se3

k,i,o).Intheﬁrstmodel(modelB1),nootherinformationthan basicscoreY_k,i,owasprovidedtothemodel,whereasinthesecond model(modelB2),boththeconﬁdencescoresV_k,i,oandthelevels ofuncertaintyUk,i,oweretakenintoaccount.Theformulaforthe

(6)

meanparameterk,i,oofthelatentvariableωk,i,owasthesamefor allmodels:

ˆ 3

k,i,o=˛k,i+o

whereastheformulasfor ˆse3

k,i,ovariedacrossthetwomodels: ˆ se3k,i,o=exp

k,i

(Baysianmodel1:B1) or: ˆse3 k,i,o=exp

k,i

_· Vk,i,o (Baysianmodel2:B2) where:

(i)k,iisarandomeffectusedtoparametrizeSDlevel;it fol-lowedaGaussianpriordistributionof

ℵ

(,) wherehada Gaussianpriorcenteredon0with5asSDandhadarelatively non-informativeprioruniformdistributionbetween0and100;

(ii)␣k,iisthemeanrandomeffectestimatedforthelatent vari-ableforeachESiandETk,withanon-informativeGaussianprior distributionof

ℵ

(˛,˛) where␣and␣hadthesamepriorsas and;

(iii)oisarandomobserveroeffectonthemean,chosenhereto beconstantacrossETsandESs,anddrawnfromacommonGaussian distributioncenteredon0andwithSDobs:

(o)_o∼

ℵ

(0,_obs)

whereobshasthesamepriorasthek,is.

(iv)conﬁdence score _V·_k,i,o is theproduct of the conﬁdence scoresofeachETandES(VET

k,oVESi,o),restrictedtoitsorder(from 1to6);

(v)theconﬁdencemultipliers (_i)1≤i≤6inmodelB2were con-structed withno constraint on their order, so that the vector summedto one(to preventconvergence issues withthek,is). Thiswasdonebyﬁrstsetting’1=1/S,i=i/Sfor2≤i≤6with S=1+6

i=2i andtheichosenfromauniformprior distribu-tionbetween0and100.Weﬁrsttriedaversionthatconstrained theorderoftheseparameterstodecreasewithi,butwemetstrong convergence issues that forced us to adopt this unconstrained structure.

Furthermore,wealsointegratedinmodelB2thedatarelatedto thecalibrationofconfidencescoresUk,i,oforspecificscoresYk,i,o cor-respondingtocases(k,i)withcontrolledlevelsofconfidenceVET

k,o andVES

i,o.Uk,i,ocouldtakepredeﬁnedvalues(0,0.33,0.5,1,and 2)andwasassumedtorepresentthestandardvariationofYk,i,o. WeintroducedtheinformationprovidedbyUk,i,o througha dis-cretedistributiononthe5possiblevaluesproportionaltoGaussian distribution

ℵ

␴

k,i,o+␳

,U

,where:

(vi)k,i,oistheestimatedSDofYk,i,oascalculatedfromtheabove equationsfor

P

Yk,i,o=y

1≤y≤5;

(vii)_is_a_positive_mean_SD_multiplier_chosen_to_multiply_the underlyingSDfromarelativelynon-informativeprioruniform dis-tributionbetween0.1and5;

(viii) _is_a_negative_number_with_a_relatively_informative uni-formpriorbetween−2.0and0.0usedtoscalek,i,otomakeitﬁt bettertothescaleofUk,i,o;

(ix)U is a meanSD withaninformative prior distribution between0and1.

We mostly used informative priors here because less-informative priors gave much noisier estimators, which was contrarytoexpectations(aswedidnotexpectthemeanofUto beabove10.0giventhevariationoftheUs,whereasweobserved suchvalueswithnon-informativepriors).

Model B2therefore incorporatedvariations of precisionand meanbetweenESandET,aswellaspotentialsystematicbiases betweenobservers.Thismodelalsoincorporatedadditional infor-mation,i.e.theconﬁdencescoreprovidedbyexpertstogetherwith

theirrawscore,aswellasa calibrationofthisconﬁdenceinan independentsurvey.Intherestofthepaper,forModelB2,ˆ3

k,i,ois meanparameterand ˆse3

k,i,oistheSE.

Thesetwo modelsplus threeadditionalmodels(see Supple-mentaryMaterials)wereﬁttedwithStanfromR3.3.0throughthe rstanlibrary(2.9.0-3)(StanDevelopmentTeam,2015).Weused3 MarkovChainswithatotalnumberofiterationsof100,000each, aburn-inlengthof20,000andathinningparameterof20. Mix-ingwascorrectandhad12,000outputvaluesforeachparameter, withanumberofeffectivevalues(CODARlibrary)above11,000for Model1andabove3500forModel2(fortheworstparameters).

Basedontheposteriorsamplesofparameters,wecalculated theposteriormeanandSEsofthemeanacrossobserversofseveral ES(ontheoriginalscaleofY)providedbythedifferentland-cover typesstudied.

3.4. Resampling

Inordertoestimatetheimpactofexpertpanelsizeonthe cen-tralanddispersionparameters,weusedabootstrapsubsampling method.Wesimulateddifferentexpertpanelsizesbysubsampling eachdataset.Weﬁxedthesizeofthepanel“n”between2andthe actualnumberofexperts(30fortheRNP-SEwetmatrix,23forthe RNP-BPmatrixand17fortheRNP-SEallmatrix),andforeachsize wegeneratedsampleswithreplacementfromtheactualdata.At eachiteration,severalstatisticalparameterswerecomputed.This approachenablesustoevaluatethevariationofthestatisticsgiven smaller-than-actualsamples.Weusedittotrytoidentifya mini-malsizeoftheexpertpoolneededtoobtainareliableestimateof theﬁnalscoresandasmallSEofthismean.

4. Results

4.1. Centralvalues,capacitymatrixscores

BasedontheRNP-SEwetdatasetthatweanalyzedinmoredetail thantheotherdatasets,weobservedthatthedifferentstatistical methodsusedtocomputecentralparameters(mean)providedvery similarresults.Table1,whichpresentsmeanandSDofallexpert scores(153scores×17expertscores;ﬁrstcolumn)andofthemean scoresofourapproaches(meansofthe153ﬁnalscoreswiththesix differentcalculations),showstheproximityoftheirmeanresults clearlyvisibleataround3.3withsmiliarvaluestothethird deci-malplaceforallexpertsscores,Meanofaveragesscores,Meanof bootstrapscoresandMeanBayesianscores.Themeanofallexpert scores(calculatedwithnormaltheorystatistics)is3.363,andSDat 1.368(Table1)withmedianat4.Forthemeanofaveragescores (meanofthe30expertscoresforeachofthe153ET×ES),theSD is0.967.

TheSDonallmeans’scoreshasthesameproximityofresultsas themeansexceptforallexpertsscoreswhichshowhigher varia-tion.

SpeciﬁcET/ESscores(seeTableS.2inSupplementary Materi-als)showforexamplethemean␮1

i,o whichranksfrom1.40for ES10“Contributiontofreshwater”inET8“Crops”to4.93forES17 “Knowledge,scientificandeducational”inET6“Marsh,bogsand reedbeds”.Examinationofmeansbetweenapproachesandspecific ESandETfindsnodistinctionbetweenmeansofthe6calculations. 4.2. Scoresvariability

We examinedmeansof SEsofthescores(basedonthe153 ES_×ETcombinations)estimatedfromeachmethodaspresented inFig.1andfoundthatvariabilityinmeanscoreswaslowestfor weightedscores(0.168)andhighestfortheBootstrapscoreswith

(7)

Table1

Statistics(meanandStandardDeviationSD)using3differentmethods:Usual,BootstrapandBayesian.Usualreferstotheuseofstandardparametricstatistics.Bootstrapis basedonresamplingandMonte-Carlomethods.Bayesianisbasedonheteroscedasticorderedprobitmodels.

Usual Bootstrap Bayesian

Onallexpert scoresYk,i,o Meanofaverage scores ˆ ␮1 k,i Meanofweighted scores ˆ ␮1 k,i Meanof Bootstrapscores ˆ ␮2 k,i MeanofBootstrap

scoreswithconﬁdence

scores␮ˆ2

k,i

MeanBayesian

scoresˆ3 k,i

MeanBayesianscores

withconﬁdencescores

ˆ 3

k,i

Mean 3.363 3.363 3.371 3.363 3.298 3.363 3.365

SD 1.368 0.967 1.006 0.967 0.903 0.932 0.935

Fig.1.Meansofstandarderrors(SE)ofthedifferentmethods.

conﬁdencescore(0.195).Thethreeapproacheswithoutconﬁdence scoresharedsimilarvariabilityinmeanscores,ataround0.173.

ThebootstrapSEwithintegrationofexpertconfidenceindex providedthehighestvalues(14%higherthanwithoutconfidence index),asexpectedbyincludinganothersourceofvariability(i.e. theindividualexpertrangeofvaluesbasedonconfidenceonthe rawscore). Thisdoesnot mean thatthis bootstrap is less pre-cisebutthat,byconstruct,itintegratesmoresourcesofvariability thantheothermethod.TheBayesianmodelwithconfidencescores alsoincorporatesintra-expertvariabilityandprovidedhigher vari-abilityinmeanscoresthanwithoutconfidencescore(0.1794vs 0.1746).

Asexaminingthewholecapacitymatrixindetailissucha cum-bersoemtask,weproposetofocusontheSEofES9“Domesticfood andtheiroutputs”foreachET.TheSEofBootstrapscoreswith con-fidencescoreshasthehighestvaluesof8–9oftheSEpresented inFig.1.However,therearenoclearlyvisiblepatternstoemerge betweentheotherfiveapproaches.TheresultspresentedinFig.2 forET8(“Crops”)aremoredispersedthanforotherET.Forthis spe-cificET,theSEofBayesianmodelsarelowerandtheSEofweighted scoresarehigherthanwithotherapproaches.Whenwelookcloser attheET8×ES9scores(availableinSupplementaryMaterialsTable S.2),themeanofaveragescoreis4.73andthemeanofconfidence indexis4.37.

4.3. Effectofexpertconﬁdence

DuringthedifferentESassessments,severalexpertsaccepted tofillthematrixonlywhenofferedaconfidencescoregivingthe opportunitytoexpresstheirsself-confidenceontheirscores.Inthe RNP-SEwetmatrix,meanconfidencescoreperexpertshowsatrend towardshighconfidencescoresfor10expertsoutof30(medianat 3)whiletwoexpertstendedtohavelowconfidencescorewitha

medianat1.ThetableS.3(SupplementaryMaterials)presentsVk,i,o inthetableandthemeansofeachVES

i,oandVETk,oinmargin. ThemeanconﬁdencescoreofallexpertsforallETandES(noted VET

k,oandVESi,o,respectively,andratingbetween1and3)is2.23 witheachconfidencescore(1,2and3)representedat20%,37% and43%, respectively.Meanscore ofallESwas2.24andmean scoreofallETwas2.21.Amoredetailedanalysisshowsthereis nouniformprofileforETorESexceptforculturalserviceswhere eachconfidencescore(1,2and3)representedat12%,28%and60%, respectively.

Theresultsfromadditionaldirect surveypresented inFig.3 showtherangeandmeansofvariationinexpertscores(the lev-elsof uncertainty Uk,i,o) related toeach conﬁdence score (with Vk,i,o=Vk,o×Vi,orecodedonthe1-to-6scaleusedinthedifferent models).

Notethegradualdecreaseinscorevariationasconfidencescore increases.ThevariabilityoftheESscoreerrorsassociatedwith con-fidencescoreswashigherforlowerconfidencevalues(1–2–3)than forhigherconfidencevalues(4–5–6).ThehighvaluesofVk,i,oare relatedtohighconfidenceoftheexpertsontheirsscoresandthus tolowervariationintheirfinalscores.Therewasnosignificant dif-ferencebetweenscoreserrorsassociatedwithconfidencescore5 and6.

4.4. Effectofexpertpanelsize

Expertpanelsizehadaneffectontwoaspectsofvariability. First,ithadaneffectonthestabilityofthefinalscoreobtainedfrom differentsetsofresampledsame-sizetables:SDofresampledfinal scoresdropssharplyfrom2to10(Fig.4)butthenremainsrelatively stablebeforeflatteningafterasizeof15forthethreematrices.This meansthataboveanexpertsizepanelof15experts,thevariation offinalscoreobtainedfromdifferentpanelcompositionsisstable.

(8)

Fig.2. Standarderrors(SE)ofES9“Domesticfoodandtheiroutputs”fordifferentland-covertypesET1:Wetforest,ET2:Broadleafmonospeciﬁcafforestationinwetlands, ET3:Wetmeadow,ET4:Pondsandpools,ET5:Miningsubsidencepondsandquarrywaters,ET6:Marsh,bogsandreedbed,ET7:Wetlandfallows,ET8:Crops,ET9:Green space.

Fig.3. Meanandstandarderrorsofthevariationinexpertscores(Uk,i,o)relatedtoeachconﬁdencescore(Vk,i,o).

ThetotalreductionofSDisapproximately0.2(Fig.4),i.e.about16% lowerwhenwehave15expertsinvolvedcomparedto2.

Second,wetestedtheeffectofexpertpanelsizeontheSEsof ﬁnalmeanscore(theprecisionofestimates).Asexpected,theSEs decreasewiththesamplesize.Thebootstrapestimatesshowlower valuesthanusualstatisticswhenthesamplesizeisbelow10–15 experts(Fig.5).Allthreedatasetsshowsimilartrends.However, evenwithanexpertpanelsizeat30,westilldonotreach stabiliza-tion.

5. Discussion

5.1. Finalscoresandvariability

Themeanisthemostwidelyusedstatisticforcomputingaﬁnal scoreinthecapacitymatrixapproach,butsomeauthorsadvocate using median scoresinstead (Kopperoinen et al., 2014). Differ-entkindofweightscanalsobeappliedtoobtaintheﬁnalscore. Dependingontheapproach,weightingcanbeappliedduringthe

extrapolationatlargergeographicscale,asinHermannetal.(2013) whochosetoweightscoresbytheareaofbaseunit.Koschkeetal. (2012)electedtoapply“explicitweightsasthe importanceofthe variousecosystemservicesmightdifferwithrespecttothecontext,the includedstakeholders,andtheinvestigatedregion”.Theseweightsare linkedtospecificmethodologicalchoices.Here,wedidnotapplyES orETdependentweightingsaswewantedtoequallyconsiderevery ESandET.Weusedaweightbasedontheconfidencescores,but theupshotisthataparticipantwithmoreconfidencethananother givesabetteroramorerealisticestimateofthehabitat’scapacity toprovidetheES.

With30 expertsinvolved intheRNP-SEwetdataset,the dif-ferentstatisticalmethodsusedtocomputeparametersprovided verysimilarresults. Theresultsshowonlymarginaldifferences infinalscoreswhenweintegrateconfidencescoreorthe uncer-taintyunderlyingconfidencescoresingeneralobservationofthe data(meanofmeans)oratanESlevel.Indeed,with30experts involved,thefinalscoredidnotchangemuchaftertakingthe con-fidencescoreintoaccount.Thevariabilitiesoffinalscoresarealso

(9)

Fig.4.Effectofexpertpanelsize(numberofparticipants)onthevariation(SD)ofmeansofaveragescores.

Fig.5. Effectofexpertpanelsize(numberofparticipants)onthemeanofSEsofES×ETscores(averageofthewholecapacitymatrix)usingusualstatistics(SE)and bootstrappedstatistics(SEBoot).

quitesimilar,exceptforbootstrapscoreswithconﬁdencescores whichhadhighervariability.Theincreaseofvariabilityiscoherent withtheadditionofinnerconﬁdenceordoubtofeachexpertin termsofvariabilityofscores.

RegardingtheBayesianapproach,thevariabilityissimilarfor thetwomodelsconsidered,whichmaybeduetothefactthatboth integrateanobservereffect.Theobservereffectmayintegratesome systematicbiasofeachexpertwhenprovidingtheirscores. Never-theless,theﬁnalresultsofthetwoBayesianmodelsarecloseto thoseoftheothermethods.However,theBayesianframeworkhas thepotentialtodeliversubstantiallymoreinformationthanjust centralvaluesanderrors—itcanclearlyintegrateallthe informa-tionandprovideestimatesontheseotherpiecesofinformation. Toillustrate,theBayesianmethodcanreachanestimateofthe variationofprecisionofthelatentprocessasafunctionof conﬁ-dencescore.Ifwehadobservedstrongsystematicbiasesbetween observers—whichwasnotthecasehere,asthemeanestimateof

obs wasaslowas0.10—Bayesianmethodswouldhavemadeit possibletoprovideestimatesofSEsthatwouldhaveremovedthese biases,whichothermethodsareunabletodo.

Given the amountof information contained in our capacity matrices,it wasnotpossibletoseparatelydetail alltheESand ETcombinations,sowehavedetailedoneexamplecombination here,andprovidedallthetablesasSupplementaryMaterialsfor interestedreaders.ForthespecificES×ETcombinationslookedat here,theresultsofET8×ES1scorescanbeexplainedbythenature ofthisscore.“Crops”haveafarhigherpotentialcapacityto pro-vide“Domesticfoodandtheiroutputs”thanotherEStypes.The variabilityofitsscoreislowduetothelogicalgeneralagreement onthispotentialcapacity.TheBayesianmodelsreflectthis gen-eralagreement“better”asthelowestSE.Onlytheweightedscore hasasurprisingresultcomparedtoothermethods,whichmaybe explainedbythefactthattheconfidencescoreislessareflection ofthevariabilityofeachparticipantthanareflectionofthe

(10)

uncer-C.S.Campagneetal./EcologicalIndicators79(2017)63–72 71

taintyunderlyingconﬁdencescoresexpressedthroughtheresults oftheadditionalsurvey.

5.2. Importanceofexperts’conﬁdenceintheirscores

Jacobsetal.(2014)recommendedfollowingtheIPCC(2014)or MillenniumEcosystemAssessment(2005)methodswith partici-pantconfidencereportedwithfivelevelsofconfidenceorassessing thereliabilityofoutcomesinasimilarstandardizedmanner.Inour case,askingforsuchspecificconfidencescoreswastoomuchof aburdengiventhetimeandthoughtalreadyinvestedtofillout thecapacitymatrix.Additionalinformationlikeaconfidencescore hastobesuitable,understandable,andonadifferentscaletothe participants’score(5levels).

Manyparticipantsexpressedtheimportanceofhavinga con-fidencescoreinthecapacitymatrix toletthemmoderate their self-reliabilityontheirownknowledge.Analysisoftheconfidence score showedthat 80% of theparticipants felt “comfortableor moderatelycomfortableontheirscores”.Itisimportanttoadda confidencescoreinaparticipatoryprocess,notjustas supplemen-tarymaterialsbutalsotobeabletorecalibrateitforintegration intotheanalysis.Weconsequentlychosetousethreelevelsof con-fidenceforETandthreelevelsofconfidenceforES,whichwhen combinedprovidea6-levelconfidencescoreforeachETandES. Thisapproachprovedtobeflexible,workable,andeasyforall par-ticipantstounderstand.

Theadditionalsurveysupportsthehypothesisofacorrelation betweenconfidencescoreandSDofthefinalscore,andshowsthe negativerelationshipbetweentheconfidencescoreandthemean ofscoreerrors.Ourconfidenceindexcanthisbeusedasaproxyof theerrorsineachexpert’sestimatescores.Alimitationhereisthat wehavenotexploredallthepossibilitiesoftheBayesianmodels. Wewereabletoanalyzethenoisearoundexpertscoreswithother approachesbutwechosenottoreporttheresultsinthispaper. 5.3. Howlargeshouldanexpertpanelbe?

Withoutdistinctionbetweenthethreematrices,thevariability oftheﬁnalscoresisstableafter15expertsinvolvedandtheSEin theﬁnalscoresdecreasedwithincreasingexpertpanelsize.The bootstrapapproachalsoincreasestheprecisionbyreducingthe variationwhenpanelsizesaresmall.

Theseresultsservetoexplainourlowdifferencebetweenmeans ofthethreeapproachesandwithorwithouttakingintoaccountthe conﬁdencescore.With30expertsinvolved,wereachastablemean thatmaythusironoutdisparities.

Thisresultisinterestinginpracticeasitistheﬁrsttimethata recommendationregardingpanelsizeisprovidedforparticipatory ESassessment.Having alargepanelisgood,butsmallerpanels canstillprovidereliableestimates.Itshouldbestressedthatthe caseconﬁgurationhereispanelcompositionbasedonexpertsand scientistswithexpertiseinecosystemsand/orESinthestudied region.Otherstudiesareneededtoaddressthequestionofpanels orsurveysbasedonnon-specialists.

Thestabilityofthescoresexpressesaconsensusofthepanel onthepotentialcapacityofagivenETtoprovideagivensetof ES.Thereisstillaneedtofurtherexplorethelinkbetweenthis consensusscoreandactualpotentialcapacity.

6. Conclusion

Thetwopartsofthis paperbringcomplementaryanalysisto helpidentifywhatmethodstouseorhowmanyexpertsareneeded. Thesequestionsalsodependontheleveloferroracceptedandof coursethenumberofexpertsthatcanviablybeinvolvedinthe participatoryapproach.

Forarestrictednumberofparticipants,itisbettertouse boot-strapestimateparametersinordertoincreasethestabilityofthe finalscoreanditsvariability.Ifmorethan15expertsareinvolved, theusualstatisticsissufficient.Indeed,with30expertsinvolvedin ourmatrix,finalscoresshowedlittledistinctionbetweenthethree approaches,aswehadreachastablemean.

Bayesianmethodsaremoredemandingbuttheyalsogivemore complete results.Bayesianmethodseffectivelyrequirea degree ofexpertisein howtowrite,ﬁtand interpretBayesianmodels. However,theydoenabletointegratedifferentkindsofinformation (here,theexpertscores,conﬁdencescoresandtheirsuncertainties werehighlighted)inanintegratedstatisticalmodellingframework (Gimenezetal.,2014)aswellastoestimatealltheparameters associatedwiththesevariables.Inparticular,Bayesianmodelshold greatpotentialiftheaimistogainamoreinferentialpointofview. Forthesakeofconsistenceandconcision,thispaperdidnotexplore allthepossibilitiesoftheBayesianmodels.

Forparticipatoryprocesses,werecommendaddingaconfidence score,onthecaveatthat incorporationofa confidencescoreor theuncertaintyitcarriestothefinalscoreinthecapacitymatrix depends on the SE accepted and the method used. We found herethatconsideringconfidencescoreinfinalscoreinducedonly marginaldifferences.

Acknowledgements

WethankJean-MichelSallesforhisinputandcommentsonthe paper.ThisstudywassupportedbytheNaturalRegionalParkof Scarpe-Escaut.(PNRSE),andwethankGeraldDuhayonand Tan-guiLefortfromthePNRSEfortheirsupport.TheRNP-BPmatrix is part of a 2017 PhD project by L. Tschanz titled“Towards a socio-ecologicalobservationforaglobalapproachtobiodiversity challengesofinregionalplanning”attheIMBE,Aix-Marseille Uni-versity.ThisworkwasfundedbyIrstea,thePNRSE,thePNRBP,and theInstitutINSPIRE.

AppendixA. Supplementarydata

Supplementarydataassociatedwiththisarticlecanbefound, intheonlineversion,athttp://dx.doi.org/10.1016/j.ecolind.2017. 03.043.

References

Burkhard,B.,Kroll,F.,Müller,F.,2009.Landscapes’capacitiestoprovideecosystem services–aconceptforland-coverbasedassessments.Landsc.Online,1–22,

http://dx.doi.org/10.3097/LO.200915.

Burkhard,B.,Kroll,F.,Nedkov,S.,Müller,F.,2012.Mappingecosystemservice supply,demandandbudgets.Ecol.Indic.21,17–29,http://dx.doi.org/10.1016/ j.ecolind.2011.06.019.

Burkhard,B.,Kandziora,M.,Hou,Y.,Müller,F.,2014.Ecosystemservicepotentials, ﬂowsanddemands–conceptsforspatiallocalisation,indicationand quantiﬁcation.Landsc.Online32,1–32,http://dx.doi.org/10.3097/LO.201434. Efron,B.,1981.Nonparametricestimatesofstandarderror:thejacknife,the

bootstrapandothermethods.Biometrika68,589–599.

Fisher,B.,Turner,R.K.,Morling,P.,2009.Deﬁningandclassifyingecosystem servicesfordecisionmaking.Ecol.Econ.68,643–653,http://dx.doi.org/10. 1016/j.ecolecon.2008.09.014.

Gimenez,O.,Buckland,S.T.,Morgan,B.J.,Bez,N.,Bertrand,S.,Choquet,R.,Dray,S., Etienne,M.-P.,Fewster,R.,Gosselin,F.,Mérigot,B.,Monestiez,P.,Morales,J., Mortier,F.,2014.Statisticalecologycomesofage.Biol.Lett.10,20140698.

Haines-Young,R.,Potschin,M.,2013.CICESV4.3–ReportPreparedFollowing ConsultationonCICESVersion4,August-December2012.EEAFramework ContractNoEEA/IEA/09/003.

Hermann,A.,Kuttner,M.,Hainz-Renetzeder,C.,Konkoly-gyuró,É.,Tirászi,Á., Brandenburg,C.,Allex,B.,Ziener,K.,Wrbka,T.,2013.Assessmentframework forlandscapeservicesinEuropeanculturallandscapes:anAustrianHungarian casestudy.Ecol.Indic.37,229–240,http://dx.doi.org/10.1016/j.ecolind.2013. 01.019.

Hou,Y.,Burkhard,B.,Müller,F.,2012.Uncertaintiesinlandscapeanalysisand ecosystemserviceassessment.J.Environ.Manag.127,S117–S131,http://dx. doi.org/10.1016/j.jenvman.2012.12.002.

(11)

IPCC,2014.ConﬁdenceandLikelihoodintheIPCCFifthAssessmentReport.IPCC.

Jacobs,S.,Burkhard,B.,VanDaele,T.,Staes,J.,Schneiders,A.,2014.TheMatrix Reloaded:areviewofexpertknowledgeuseformappingecosystemservices. Ecol.Model.295,21–30,http://dx.doi.org/10.1016/j.ecolmodel.2014.08.024. Kopperoinen,L.,Itkonen,P.,Niemelä,J.,2014.Usingexpertknowledgein

combininggreeninfrastructureandecosystemservicesinlanduseplanning: aninsightintoanewplace-basedmethodology.Landsc.Ecol.29,1361–1375,

http://dx.doi.org/10.1007/s10980-014-0014-2.

Koschke,L.,Fürst,C.,Frank,S.,Makeschin,F.,2012.Amulti-criteriaapproachforan integratedland-cover-basedassessmentofecosystemservicesprovisionto supportlandscapeplanning.Ecol.Indic.21,54–66,http://dx.doi.org/10.1016/j. ecolind.2011.12.010.

Kroll,F.,Müller,F.,Haase,D.,Fohrer,N.,2012.Rural-urbangradientanalysisof ecosystemservicessupplyanddemanddynamics.LandUsePolicy29, 521–535,http://dx.doi.org/10.1016/j.landusepol.2011.07.008. Litchﬁeld,J.,Reilly,B.,Veneziani,M.,2012.Ananalysisoflifesatisfactionin

Albania:anheteroscedasticorderedprobitmodelapproach.J.Econ.Behav. Organ.81,731–741.

Martnez-Harms,M.J.,Balvanera,P.,2012.Methodsformappingecosystemservice supply:areview.Int.J.Biodivers.Sci.Ecosyst.Serv.Manag.8,17–25,http://dx. doi.org/10.1080/21513732.2012.663792.

MillenniumEcosystemAssessment,2005.EcosystemsandHumaWell-Being Synthesis.IslandPress,Washington,DC.

Seppelt,R.,Dormann,C.F.,Eppink,F.V.,Lautenbach,S.,Schmidt,S.,2011.A quantitativereviewofecosystemservicestudies:approaches,shortcomings andtheroadahead.J.Appl.Ecol.48(3),630–636,http://dx.doi.org/10.1111/j. 1365-2664.2010.01952.

StanDevelopmentTeam(Ed.),2015.StanDevelopmentTeam.

Stoll,S.,Frenzel,M.,Burkhard,B.,Adamescu,M.,Augustaitis,A.,Baeßler,C.,Bonet, F.J.,Carranza,M.L.,Cazacu,C.,Cosor,G.L.,Díaz-Delgado,R.,Grandin,U.,Haase, P.,Hämäläinen,H.,Loke,R.,Müller,J.,Stanisci,A.,Staszewski,T.,Müller,F., 2014.AssessmentofecosystemintegrityandservicegradientsacrossEurope usingtheLTEREuropenetwork.Ecol.Model.295,13,http://dx.doi.org/10. 1016/j.ecolmodel.2014.06.019.

Tschanz,L.,Tatoni,T.,Brun,J.-J.,2015.Servicesécosystémiques.Pourunegestion durabledesterritoires,pensonsenbouquet.Espac.Nat.52,27–28.

Vihervaara,P.,Kumpula,T.,Tanskanen,A.,Burkhard,B.,2010.Ecosystem services-Atoolforsustainablemanagementofhuman-environmentsystems. CasestudyFinnishForestLapland.Ecol.Complex.7,410–420,http://dx.doi. org/10.1016/j.ecocom.2009.12.002.

Vihervaara,P.,Kumpula,T.,Ruokolainen,A.,Tanskanen,A.,Burkhard,B.,2012.The useofdetailedbiotopedataforlinkingbiodiversitywithecosystemservicesin Finland.Int.J.Biodivers.Sci.Ecosyst.Serv.Manag.8,169–185,http://dx.doi. org/10.1080/21513732.2012.686120.

Willemen,L.,Burkhard,B.,Crossman,N.,Drakou,E.G.,Palomo,I.,2015.Editorial: bestpracticesformappingecosystemservices.Ecosyst.Serv.13,1–5,http:// dx.doi.org/10.1016/j.ecoser.2015.05.008.