Towards systemic change: On the co-creation and evaluation of a study programme in transformative sustainability science with stakeholders in Luxembourg

Towards

systemic

change:

on

the

co-creation

and

evaluation

of

a

study

programme

in

transformative

sustainability

science

with

stakeholders

in

Luxembourg

Ariane

Ko¨nig

Thispaperexaminesastudyprogrammein‘Sustainabilityand socialinnovation’attheUniversityofLuxembourgthatwas co-createdwithkeyexternalstakeholdersinlocalsustainability transitions.Theprogramme’saimistoequipscientistsand citizensforthepracticeoftransformativesustainabilityscience tochangehumanenvironmentinteractions.Addressing sociallysalient,complexproblemsinvitesare-conceptionof whatroleuniversitiescanplayinknowledgeproduction processesinmoreappliedandlocalcontexts.Wecritically discusstheprogramme’sambitiontoprovideaplatformfor transformativesociallearningforsustainabilityandto contributetofosteringsystemicchangeinLuxembourg.We deducedesignrequisitestoachievetheseambitions.The paperalsodiscussestheroleofdifferentformsofevaluationin effectingindividual,programmeandsystemicchange. Researchinsightsweredrawnfromdocumentaryandliterature research,concept-building,programmeimplementation, observation,analysis,andevaluationbystudentsand contributors.

Address

UniversityofLuxembourg

Correspondingauthor:Ko¨nig,Ariane(ariane.koenig@uni.lu)

CurrentOpinioninEnvironmentalSustainability2015,16:89–98 ThisreviewcomesfromathemedissueonSustainabilityscience EditedbyArianeKo¨nigandNancyBudwig

Received16May2015;Revised05August2015;Accepted06August 2015

http://dx.doi.org/10.1016/j.cosust.2015.08.006

1877-3435/#2015ElsevierB.V.Allrightsreserved.

Introduction

The persistent lack of action in response to growing evidence that our industrial metabolism exceeds the planet’s biophysical carrying capacity and thereby threatensour lifesupport systemis causingincreasing concern across the globe.The existential problems of civilization in the 21st century are complex, as they involve interactions between society and culture, the techno-sphere and the environment. Traditional

disciplinary fields of ‘normal’ science can only play a limited role in resolving such complex problems [1], especially considering the prevailing rift between the natural and the social sciences [2,3]. For these ‘post-normal’problemssociety requiresmorediversespaces forknowledge creation [4,5].

‘Sustainability science’ has been conceived as a new branchofsciencethatistotakeaccountofcomplexity and uncertaintybyadoptingasystemsperspective[6]. Monitoringandlearningarekeygoalsofsuchresearch. Increasingly sophisticated conceptions of transforma-tive sustainability science for fundamentally changing human–environment interaction are being designed and put into practice [7–9,10,11]. The appropriate sciences for sustainability will involve recognition of complexity, conflict, uncertainty andignorance. It can be argued that key to such transformative scientific inquiry isrethinkinghownewactionable knowledgeis co-created in collaborative processes. With their com-bined mission of research, teaching and engagement, universitiescanplayaleadingroleinestablishingsuch processes.

Researchthatguided theco-creationprocess andisthe basisforthispaperincludesdocumentary andliterature research in environmental and sustainability education, philosophy and sociology of science, as wellas partici-pant-observationandevaluationinindividualreportsand focusgroupmeetings.Thesewerethebasisfor concept-building, implementation and subsequent observation, analysis and reflections, leading into further concept-building,actionandreflectioncycles.

SustainabilityattheUniversityofLuxembourg

The University of Luxembourg (UL) is Luxembourg’s firstandonlyUniversity.Theuniqueattributesof Lux-embourg and of the university that played a role in shapingofthisstudyprogrammearelistedinBox1. Lux-embourgisasmallcountrywithshortpathstohigh-level decision-makers and politicians. In general terms, pro-grammecontributorshavenotedthattheyoung universi-ty is proving remarkably flexible in accommodating innovative forms of education and research compared withamature,largeuniversity,asforexamplethe neigh-bouringUniversityof Trier.

Anotherhelpfulfacetisthatinter-disciplinarityfeatures asakeyorganizingprincipleinthe2003lawfoundingthe university.However, securing externalfundingfor pro-grammesandprojectsthatexplicitlystatetheambitionof questioningprevailingsocialnormsandstructuresinview ofsustainabilitychallengesseemstobeatleastasdifficult inLuxembourgthathasaverytraditionalculturewitha prominentsaying‘mirwolle bleiwenwat mirsin’(we’d liketostaywhoweare),aselsewhere.Theabilitytodraw ontheUniversity’sowninternalfundsbyestablishinga CellforSustainabilityandendorsementof transdisciplin-ary activities bridging diverse disciplines and practice, also based on the legal remit, from the rector was a requisitefor success.

The building thetransdisciplinary activitieson sustain-abilityattheUL,includingtheCertificatewouldnothave been possible without engagement in an international networkofleadinguniversitiesastheInternational Sus-tainableCampusNetwork(ISCN).TheHeadof Sustain-abilitydevelopedtheUL’sfirstStrategicActionPlanon Sustainability (2010–2013).1 in a participatory process withstaff,studentsandexternalstakeholdersatthesame timeasactingaslead-authorfortheISCNCharter.After 21oftheworld’sleadingUniversitiessignedtheCharter theUL Presidentfollowedsuit.The actionplan’smain goal is ‘to engage students, staff and interested civil society in experiential learning about how to reduce environmental impacts and enhance social cohesion and buildcapacity to developand implement solutions bydrawingondifferentdisciplines.’Workin internation-alnetworkscontinuesto informlocalactivities.

Network-buildingat thenational,regional and interna-tional level allowed gaining visibility and recruiting an internationally and locally strong, loyal and influential teamof contributors for theCertificate. It alsobenefits fromastrongpoolofprofessionalmid-careerparticipants fromtheEU institutions.Thetri-lingual Universityhas an international student body offering global perspec-tives.

Whilst different language abilities of contributors and participantsremainachallenge,theadvantagesofstrong international contributions and transferring knowledge fromleadinguniversitiesinthisfieldelsewhereare con-sideredtoweighoutdisadvantages.Themaindraw-back isthatteachinginEnglishreducesthepotentialtohave traction with the local community. Improved ways to organize knowledge transfer to local networks seeking systemicchangeinLuxembourgisakeyareaforfuture improvements.

Thecertificateinsustainabilityandsocialinnovation

TheCertificate,approvedasanofficialstudyprogramme in November 2012, is the university’s first part-time programme open to Bachelor-, Master-, Ph.D.-students and professionals. The offering of the first course took 18monthstoprepareemployinganinterdisciplinaryteam

Box1 Uniqueattributesofthecontextualandtransactional environmentoftheUniversityofLuxembourg(UL)shaping possibilitiestobuildSustainabilityScienceandeducationforit

Luxembourg

LuxembourgisaseatofmajorEUinstitutions(PartofEU ParliamentandCommission,EUROSTAT,CourtofAuditors, EuropeanCourtofJustice)—theinternationalcommunity pre-sentsapoolofhighlyprofessionalcross-culturallysensitive contributorsandparticipants.

ThesmallsizeofLuxembourganditsgovernmentpresentshort pathstodecision-makers.

Luxembourg’sconstitutionprescribesthateducationalcontent hastobeprogrammedbylaw.

TheUniversityofLuxembourg

Createdbylawin2003asLuxembourg’sfirstandonlyuniversity Aresearchorienteduniversitywith3facultiesoffering46Bologna studyprogrammesattheBachelorandMasterslevelinnatural science,socialscienceandhumanities,lawandeconomicsand finance.Connectionofresearchandteachingisastatedpriority Article3.inthefoundinglawprescribesinter-disciplinarityasa

mainorganizingprinciple.

International:ObligatorysemesterabroadforallBachelors, tri-lingual(studyprogrammesandadministrationinEnglish,German andFrench),staffandstudentsfromover100differentcountries, under50%Luxembourgstudents.

Small:Seekingtocapat7000studentstoremainpersonable Sustainabilityisendorsedatthehighestlevel:thepresidentsigned

theInternationalSustainableCampusNetworkCharter2010(UL HeadofSustainabilitywaslead-authoroftheCharterandplaysa leadingroleinthisnetwork)

1

of tennaturalandsocialscientistsandtwopractitioners, which has been running since March 2011. After two furthercourseswerebuiltandexperimentationwithpeer group projectshadmaturedover twoyears,thepackage waspresentedandapprovedasastudyprogramme.The programme’smainstatedgoalistoserveasaplatformfor societaldebate,sociallearningandnetworkdevelopment to foster systemic transformation for sustainability in Luxembourg.

Howis‘learning’conceivedofandwholearns?

TheCertificateisconceivedasaplatformfora transfor-mativesociallearningprocess;themaingoalistoprompt scientific inquiry in diverse groups of stakeholders and experts. Theunderlyingconceptionofsociallearning is similartothatdescribedinBart[15]andWalsetal.[10]. It focuses on the relation of learning across different scalesof socialorganization–individual,group, organiza-tionalorsocietal—andbuildsonprevailingconceptions ofsociallearninginenvironmentalmanagement[16,17]). The conception of transformative learning as collective scientific inquiry is close to that described by Wals et al. [10] and is rooted in John Dewey’s work (1938) [18] and more recent interpretations of it [19]. Accord-ingly learning is a process of developing an enriched understanding and repertoire of action on problems as aresultofopenexperimentationandjudgmentofresults followingcriteriaofrationality.Thefactofknowingmore andmasteringknowledgeinadifferentmannerchanges ourrelationshipbetweentheworldandourselves. Prog-ress then buildson theevaluationof andpassing judg-ment onadirectionofdevelopment.

The mainattributes of our conceptionof sustainability scienceare (i) collaborativeinquirywith asystems per-spectivetocharacterizecomplexity;(ii)diversityof theo-ries and methods including the natural, social and practice-based sciences and humanities, that are juxta-posedinaprocessallowingforcriticalinterdisciplinarity to transformeachengageddiscipline toovercome limit-inganddivergentassumptionsandpre-suppositions(see also[20]);(iii)diversityofstakesandinterestto under-standandmakeexplicitdivergentpreferencesand prior-ities and their value bases that are united in their orientation to co-create more sustainable futures; (iv) requisites for processes to critique, judge and evaluate new knowledge emergingfromsuch processesfrom di-versepointsofview;and(v)empathy,humility,reflection required when directing attention to people, roles and relationshipsinplace-basedand issue-basedanalysis. Thiskindoftransformativelearningprocessassumesthat knowledgeisconstructedforaction,andthatlearningcan bemediatedbypractice[21].Transformativelearningfor sustainability, engageslearners to rethink and actupon how societies and individuals interact with each other and their environments. Learning often happens by

challenging boundaries of learning environments [22]. Learning is not only based on personal experience in the senseof Kolb [23],but learners—including teach-ers—need to be challenged by the experiences and perceptions of others in adialectical manner. Transfor-mative learning relies on collective learning in diverse groups, organizations or networks. In order to embrace complexity, conflict,uncertaintyandignoranceweneed todrawonpluralrationalitiesandcontradictorybehavior. Successfullearninginterventionsneedtobemanagedto ensurethatexperientialsituatedknowledgefromdiverse communitiesofpracticeismadeexplicit,communicated and understood by others. In line with Sterling [24], transformativelearningisconsideredasalife-long itera-tive process, doors to which may be opened through engagementinprojectsthatintegrateeducation,research andcivic engagement[25].

ThusintheCertificate,contributorsandparticipantsare seen as membersin a diverselearningcommunity who engagewitheachothertoshedlightonmultiplefacetsof complexproblems,activelyexchangingandlearningfrom eachotheronanon-hierarchicalsociallearningplatform.

Whatisthepurposeoflearning?

The Certificate’s overarchinggoalistoprovide abetter understandingofandrepertoireofactiononthecomplex challenges that societies, organizations and individuals faceasweapproachthelimitsofthebiophysicalcarrying capacityof ourplanet.SustainabilityandSocial Innova-tionareinseparable:Activeinvolvementofbothcitizens and scienceisnecessaryfor guidingandmonitoringthe innovations thatpromiseto protector improve our life-styles.Thestudyprogrammeofferstoolsforsocial learn-ingtotapintothecollectiveintelligenceofstakeholders and experts.Issuessuch asenergy, waterandfood pro-duction, waste and recycling, sustainable housing and transportarecovered.Thetwocorecoursesandauxiliary coursesprovideanoverviewonthemostrecentinsights from academics and practitioners relating to global changeandtotransformationallearningforsustainability. Peer group projects on complex transition problems in Luxembourg form another central aspect in the pro-gramme providingopportunitiesfor engagementin col-lective problem-basedscientificinquiryin smalldiverse groups withguidance fromexperts andstakeholders, to allowfortransformativesociallearning.Suchexperiential learningopportunitiesequipparticipantsformaking bet-terinformedand,ifdiversityismanagedsuccessfully,also formakingmorereflexivejudgmentsforeffectiveaction. Thestructure oftheprogrammeallowsthedesignof an individuallearning path(seeFigure1).

the experience of at least one semester of peer group workisconsideredessential.

Thecontentofthecourses

The Core Course 1 ‘Science and Citizens Meet Chal-lengesofSustainability’(SCCS)equipsparticipantswith conceptualtoolsandmostrecentinsightsfromacademics andpractitioners to actively engagein societal transfor-mation for sustainability in a networked world. Core Course2‘SocialEnterpriseandSocialInnovation’(SESI) exploressocial enterprisesas vehiclesfor social innova-tiontoaddresschallengesofsocialinequityand sustain-ability. Sessions address a topic usually from diverse perspectives from the natural and social science and practice. All contributors are encouraged to state their assumptionsintheoryandmethodstheyrefertoaswellas personalconvictions andmotivations to engagein their researchorprofession.Theseare requisitesto allow for critical inter-disciplinarity, which directs attention to

limitationsandcontradictionsbetweenspecific disciplin-ary approaches to generating new knowledge, explicit deliberation of resulting contradictions is however not alwaysachieved.

‘Complexity’isacentral ideaofall courses[26], uncer-taintyispresenttoalldegrees,andthereisnoprivileged perspectiveamongparticipants.Theroleofmeasurement regimesandsetsofindicatorsindefining,monitoringand trackingdynamicsanddefiningprogressandtheirroleas devicesforsteeringdevelopmentacrossvariouslevelsof socialorganizationisexploredfromdiverseperspectives [27], including from sociology [28] and practice [29]. Buildingon[30]threedifferentlogicswithwhichsocietal sustainabilitychallengesmaybetackled,andwhichoften provedifficult to reconcile in social enterprisesare dis-cussed (community/reciprocity; hierarchy/redistribution; market/competition). Inacknowledgement of the influ-enceoflearningenvironmentswehaveorganizedcourse sessionsin specialco-workingspacesthatwererecently set up to foster social and technological innovation in Luxembourg.2 Table 1 provides an overview on main contentelementsofthecourses.

Intermsofshapingcoursecontentwithstakeholdersand academics,themostdifficultchallengehasbeentogeta discussiononproblemsandtrade-offsattheinterfaceof thesilosinwhichweusuallyorganizeourknowledgeand governancesystems,forexampletosystemicallyexplore andanticipatefutureproblemsatthenexusof food-water-energysystems.Furthermore,whilst theproblem-based peergroupprojectshavebeenorganizedwiththegoalof providingspacesforintegrationofperspectivesfromthe Figure1

→ required Core Courses

SCCS SESI

Peer Group Project

Peer Group Project Auxiliary Course

4 ECTS 4 ECTS 4 ECTS 6 ECTS 6 ECTS a second semester one semester or → required → choice Current Opinion in Environmental Sustainability

Optionsforpersonallearningpaths.TheCertificaterequires20points oftheEuropeanCreditTransferSystem(ECTS)thatcanbecollected bycompletingcorecourses,peergroupprojectsandauxiliary courses.OneECTSpointcorrespondstoaworkloadofabout1hour, oralectureof45minutes.Participationinthetwocorecourses (6ECTSeach)andatleastonesemesterofapeergroupproject (4ECTS)isrequired.Asecondsemesterofapeergroupprojectcan beoptionallyreplacedbycompletinganauxiliarycourse(4ECTS).At thetimeofwriting,twoauxiliarycourseswereon‘Global

EnvironmentalChangeintheAnthropocene(GECA)andon ‘SustainabilityReporting’followingtheguidanceoftheGlobal ReportingInitiative(GRI).Individuallearningpathscanthusbemore practiceorientedbyenrollingintwosemesterofpeergroupproject (recommended),orhomeinmoreonthescientificknowledgebase (GECA),orpracticalknowledgeondevelopingorganizationalreports. Theprogrammeisdesignedtobecompatiblewithafull-timejobor studyprogramme.Studentshaveachoice:Theycanenrolltoobtain theCertificateassecondqualificationinparalleltopursuingtheirmain degreewhilstattheUniversity.Alternativelytheycantakeindividual coursesasoptionalcoursesthatcounttowardstheirmaindegree.

Box2 Mainlearningoutcomesofthecertificateinsustainability andsocialinnovation

Toapplysystemsthinkingtounderstandthecomplexityofsociety, environmentandtheirinteractions.

Torespecttheconflictingperspectivesonanissuethatareheldby diverseexpertsandstakeholders,stemmingfromdiversityin experience,valuesandworldviews.

Torecognizeuncertaintiesandtensionsarisingfromthegulf betweenlocalandglobalperspectivesandmodesofinquiry. Skillsinnegotiation:respect,listening,givingandtakingtofind

mutuallyacceptablesolutionstocomplexproblems.

Anappreciationofalternativeformsofsocialorganizationand enterpriseforachievingasustainableeconomicexchangesystem. Todevelop‘citizenscience’approachesandtechniquesfor

creativelyintegratingthesocialandscientificemphasesofthetwo phasesofthecourse.

Toengagescienceandscientistsproductivelyinsociallearning processeswithdiversegroupsofstakeholdersforconcerted actiononlocalissuesofenvironmentandsustainability.

2_{Forexample,the15351CcreativityhubDifferdange:http://www.}

naturalandsocialsciences fromthecoursesthishasnot alwaysbeenachieved.

Howistransformativeproblem-basedlearningorganized?

This conceptionof transformative learning is to thatof Walswhodescribesitas‘openinguptoandrelatingina different mannerto diverse waysof knowing, based on interactingwithothersandtheworldaroundyou’[31,32]. Thus,theemphasisinthedesignofpeergroupprojectsis on creating spacesto work with contradicting perspec-tivesandframingsfromtheoryandpracticeanddealing withresultingcontroversiesonhowbesttocreate action-able knowledge.The projects stage collectivescientific inquiryrelyingoncyclesofactionandreflection.Diverse learning environments and situated scientific inquiry includinginterviewsandobservationalstudieshave prov-en essential.They enable thelinking of identity, com-munity,andplaceandconsidercitizenscienceasatoolfor reconnection to concrete situated problems and local stakesand interests[12,33].

Peer groups were encouraged to draw on methods to draw systematically on collective intelligence (see Table 1. Section on ‘Methods’).The projects were co-created together with academics and stakeholders who are active in transition practice in the public sector, private sector or organised civilsociety in Luxembourg

andsomeofwhomprovideguidancetotheprojects.(See overviewonpeergroupprojectsTable2.)Participantsin eachgroupwereselectedbasedontheirmotivationand diversity within the groups with respect to age and expertise/disciplinarybackgrounds.

Peergroupprojectsareparticipant-ledinthatthegroup remit initially has a broad scope, and the group has to develop their own problem-framing and specific ap-proach. For example, peer groups ondemocratizing re-newableenergyfirsthadtoagreeonanangleofinterestto all group members, for example how the group may developresourcesofinterestforestablishmentofcitizen cooperativesintheenergysector.Thegroupsareaskedto drawtogethermoreabstractacademicliteratureresources on social innovation and the structure of the energy sector,legalandregulatorydocuments,aswellaspractical informationonlocalemergingcitizencooperatives, spe-cificlocalproblems,andinformationfromdirect interac-tionandguidanceonneedsbyactivemembersofthree energycooperatives inLuxembourg.

The main challenge encountered in organizing peer groupprojectsistheresourceintensityofguidinggroups toestablishasociallearningprocesswithstakeholderson complex problems. First, time-wise, one semester pro-jectsprovedjust sufficientformostgroupstodevelopa Table1

Maincontentelementsofthecourses

What Course

Topics

(Foreachtopicwestagediscussion ofcompetingtheoriesandpractices)

Transformativesustainabilityscienceassociallearningprocess Competingconceptionsoftheeconomy,progressandtherole ofsocialenterprise

Equityasadimensionofsustainability

Competingconceptionsofscienceandprogress Cognitionofcomplexity(challenges)

Theroleofmeasurementregimesassteeringdevices Energytransition

Agriculturaltransition

Towardsimprovedwatergovernance

SESI&SCCS SESI SESI SCCS SCCS SESI&SCCS SESI&SCCS SCCS&GECA SCCS&GECA

Analyticconceptualtools Systemsthinking(largelybasedon[26,43])

TheMultiLevelPerspectiveonSocio-TechnicalTransitions (largelybasedon[44])

Competinglogicsinthesocialandsolidarityeconomy[30] Thebusinesscanvasmodel[45]

ScenarioanalysiscollaborationwithOxfordScenarios Programme[46,47]

Uncertaintycharacterization(seewww.nusap.netfor resources) SESI&SCCS SESI SESI SCCS&GECA SCCS&GECA SCCS&GECA

Methods Methodsfortappingintocollectiveintelligenceinworkshops[48] Co-DesignProcesses[49]

Presencingtechniques[50] Multi-criteriaanalysis[42] Citizenscience[10_]

SustainabilityReportinginOrganizations(largelyfollowingthe GlobalReportingInitiative(GRI)

commonframingandunderstandingofthecomplexityof theissueatstakeandcometotermswithdifferencesin values and view pointsin thegroup and have tensions surface,but insufficient to a group reflective process in thissetting.Usuallytherewas insufficienttimeto over-cometensions productively.Reportsafterone semester oftenjustmapped theissues,butprovided littlelinkto theorydiscussedin class andconceptualanalysis.Some moresuccessfulgroupsdemonstratedsignificant achieve-mentsin particular where mentors facilitated thesocial groupprocesstounderstanddifferencesandfindcommon groundinthebeginning.Sincetheyear2014/2015peer groupsnowrunforoneyearinsteadofjustonesemester. Teachingmodules onsystematicdocumentary research andprojectmanagementarebeingadded.Non-academic experts and mentors are invited to the Corecourses to

guidethegroups to drawonrelevant theoryand meth-ods.The challenge of resource intensity of providing guidance for transformative learning opportunities remainsanotherkeyareaofimprovement.Aninnovative evaluationapproachhasbeenimplementedinanattempt tomanagediversityandtensionsingroupsandtodeploy resourcesofferingguidance moreeffectively.

Evaluation

and

judgment

Evaluation of such transformative learning should thus servepurposesofinformingfurtherlearningand account-ability[34].Fouroverlappingobjectivesofevaluationin theCertificatearedescribedinFigure2.Forevaluationto effectlearning,eachindividualparticipantisencouraged tokeepareflectivediary,andintheself-assessmenthasto reflectontheirownlevelofengagementandlearningat Table2

Overviewonpeergroupprojects

Peergrouptopics Participants/semesters Stakeholderinteraction Finalreports

Democratizingrenewableenergy(often focusedonbuildingenergycooperatives)

35/6 Meetings,interviews,1workshop 4

Newformsforcooperativeandsocial housingschemes

8/2 Meetings,interviews 2

Socio-technicaltransitionin

municipalities—theroleoflocalindicators

33/6 Meetings,interviews,Survey, focusgroups,2workshops

4

Promotingwastereduction,reuse,and recyclingonandbeyondcampus

17/4 Meetings,interviews 4

WaterGovernance 11/2 Meetings,interviews,2surveys 2

Figure2

Are complexity, uncertainly, contradictions and value

conflicts considered?

Outcomes/Impacts: Shared knowledge for

concerted action? Systems analysis from

diverse porspectives that link theory and practice? Rigorous participatory scientific inquiry? Quality control? Reframing?

Evaluation

Sustainability science?

3. To certify what has

been learned as evidence for

development/progress 4. As diagnostic tool for the organization of learning

Transformation of self and environment?

Current Opinion in Environmental Sustainability

Process: Content:

Context:

theendofeachsemester.Afinalsynthesisreportasksfor one section (which is not considered in grading) with reflection on impacts of the course on personal and professionalplans.Abouttwothirdsofthereportsprovide evidence of individual transformative learning, as sug-gestedbythetwoextractedstatementsbelow(seeBox3). In2015anevaluationbasedonsurveyquestionnaireswas added.

Theevaluationofpeergroupprojectsreliesonjudgments at several levels: Self-evaluation by participants; peer groupself-assessmentinagroupdiscussion;andevaluative feedbackbythe‘peerprojectsteeringgroup’,thatis,the facilitators,togetherwithkeystakeholdersintheproject. Evaluationthroughmultipleperspectivesisdeemedmore valuableforlearningthanjustdrawingononeperspective. First,attheendofeachsemestera360degreefeedbackis organized,whereeachparticipantisaskedtoevaluatetheir levelofengagementandthatofallothermembersinthe group.Thepeergroupthenreflectsontheirwork,pooling individualreflectionsinagroupjudgmentofstrengthsand weaknessesoftheirwork.Thesereflectionsareincludedin thepeergrouppresentationandinthefinalwrittenreport, Theseinturnareevaluatedbygroupfacilitatorsandthe coursecoordinator.

Theexternalstakeholders’partoftheevaluationthrough theprojectsteeringgroupisessentialforasociallearning platform. They include key actors in the Luxembourg transition and social innovation movements, such as a businessangelwhoalsothechairtheboardofthe Euro-pean Venture Philanthropy Association, the founder of Luxembourg’slargestrenewableenergycooperative,and theco-foundersofLuxembourg’srapidlygrowing transi-tionmovement,aswellasofficialsfromtheMinistryfor Sustainable Development and STATEC and EURO-STAT(Table3).

Theprojectsteeringgroupiscomposedofallpeergroup project facilitators in any given academic year, and of otherstakeholdersintheCertificatewhoareinterestedin past,presentorfuturepeergroupprojects.Thesteering groupmeetsandevaluatestheseanddevelopsfeedback as a team. On this basis workfor the next semester is planned.The‘peerprojectsteeringgroup’alsoprovides feedbackonthestudyprogrammeandoutcomesinmore general terms, from their perspective on how well the programmemanagestodevelopaninterfacewith transi-tion initiatives in Luxembourg and how policy makers andentrepreneursaretryingtofostersystemicchangefor sustainability.Allfeedbackisconsideredinthedesignof the next year’sprogramme, including course work and peer groupprojects.Selectedquotesonthe transforma-tive impacts of the study programme from participants andacontributorarepresentedinBox3.

Table3

Multi-levelevaluationandfeedbackapproach

Participants Contributors/Stakeholders

Courses -Evaluationandself-evaluationofpersonallearning inthepartIIindividualreports(nomarksonthis) -Over80reflectivereportsoncoursesinfiveyears -Web-basedquestionnaire

-Focusgroupevaluationinclosingsession

-Focusedgroupdiscussionandfeedback collectionincourseplanningmeetingseach semester—documented

-Interactionwithindividualsonissuesofconcern

Peergroupprojects -21projectsoverfouryears peergroupinterimreports

-Conductofanexpertpanelevaluationoffinal projectpresentations(academicsandstakeholders) -12formalreportssincetheofficialstartofCertificate -Self-evaluationofengagementandlearningand evaluationaspartof360feedbackatendofeach semesterovertwoyears

-Groupevaluationofprojectaspartoffinal presentationandreport

-Focusedgroupdiscussionandfeedback collectioninpeergroupplanningmeetingseach semester—documented

-Interactionwithindividualsonissuesofconcern -Overfouryearsweheld7focusgroupmeetingsof contributorstodiscussmeritslimitations improvementsattheendofeachsemester Box3 Quotesfromparticipantsandcontributorsinevidenceof perspectivechangesandtransformativelearning

‘Thiscourseimprovedmycriticalthinkingenormously.’StudentBA Economics

‘Itwasagreatexperiencingsuchdiverseperspectivesstriving towardsacommongoalduringthecourse.Thischangedmyideaof sustainability,whichInowconceivelikeapuzzlewhereyouneeda lotofdifferentpiecestogetabetterpictureofacomplexsituation.’ Ph.D.candidate,EnvironmentalPsychology

Thesequotespresentevidenceoftransformativelearning atalllevels.Individuallevelparticipant’sstatementsB.A. andPh.D.andacontributorwhoalsohighlightsbenefits fromparticipationin thisplatformto gainnew perspec-tives. At the level of the programme organisation, the Certificatehas benefitted and changed progressively in consecutive years, and we actually althoughaiming for relational change come closer to organizing the course aroundcompetencesoftenstatedaslearningoutcomesin sustainabilityeducation[35].

However, impact and outcomes remain ill-defined and difficulttoreporton.Societaleffectsforsystemicchange will be similar to those described for participatory sus-tainabilityresearch [36],including thewholerange of quality products, knowledge gains, increased decision-makingcapacity,enhancednetworksand transformation-al changes. In further development of the evaluation approachmoretimewillneedtobeinvestedondefining whatisto beevaluated andhow inparticularasregards thecontributiontosystemicchange.

Areas

for

improvement

and

outlook

The Certificate experiments with a novel approach for combining within a study programme the practice of transformative social learning with scientific inquiry in diverse groups of engaged students, professionals and academics, similar to conceptions of the new role of the University in the face of sustainability challenges advancedby Sterling [37]and Barth [15],p. 166. The Certificateintends toprovideaspaceforsociallearning engaging scientific inquiry, to complement activities drivenbyorganizedcivilsocietyorgovernmenttofoster systemicchangeforsustainability.The emphasisinthe Certificateisplacedondrawingoncriticalresearchandon improvedwaystocaptureandjointlyevaluateoutcomes andimpactsfromdiverseperspectives.

Theapproach reliesoncarefulbuildingof anetwork of diverseparticipantsandmanagementofthisdiversityto gain a shared understanding of very diverse facets of complexsustainabilityissuesinapluralistsociety.Italso matters that learningexperiences are embedded in di-verselearningenvironments,includingformal,informal, non-formal educational settings. Programme design shoulddirectattentiontothecomplementarityof physi-cal, institutional and virtual learning environments [8,33,38].

Theareasforimprovementidentified inthispapermay however also point to the limits of a mere study pro-grammetocontributetosystemicchange.Thispaperhas pin-pointedfourmainareasfor improvement. The first issueistofindimprovedwaystogiveaprogrammewith international perspectives and contributions traction in local settings on local issues; in view of challenges of extrapolation of global abstract insights exasperatedby

challengesof working in a multi-lingual and multi-cul-turalsettingisakeyareaforfutureimprovements.The secondchallenge relatesto overcomingknowledge and governancesilos more effectivelyto allow formore tar-gettedsocial innovation, for example to jointly explore problemsandanticipatefutureproblemsatthenexusof food-water-energy systems. Improved conceptual and methodological approaches are required to integrate knowledgefromdiversedisciplinesandpractice,aswell asproblem-solvingwithcriticalperspectives.

Thirdtheresourceintensityoforganizingtransformative learningopportunitiesin theformofpeergroupworkis notable,inparticularifthesearesetupinisolationfrom research projects.Thishasled to acritique of arisk of developinganelitiststudyprogrammethatisnotinreach with resources available at a typical main stream large scaleuniversityasinGermanyorFrance.Finally,further formalizing qualitative and quantitative evaluation dependsonimproveddefinitions of potentialoutcomes andimpacts.Todate,iterativejointcritiqueand evalua-tionofthisprogrammehasprovenessentialforfostering the programme’s potential to contribute to systemic change by producing salient knowledge for transition initiativesin.Thereishoweverfurtherroomfor improve-mentofthis.

Moreformal institutionalspaces for transformative sus-tainability science projects on salient and existential issueswith keystakeholdersare required,formalization isexpected to stabilizetheseas learning environments. Approachessuchasthosedescribed byEvanset al.[39] and Rosenberget al. [40] in this volume seem comple-mentary to the approach of building a dedicated study programmetobuildcapacityforengaging scientistsand citizensinsustainabilityscience.Possibleformsforsuch institutionalspacescanincludemulti-yearresearch pro-jects. The time-factor should not be under-estimated, experiencesuggestsgroups requiretimeand asafeand guidedlearningprocesstoopenuptobeingcriticizedina constructivemanner,thisiswhyformalizedinstitutional spaces are required to provide more stability for social interactions amongst diverse stakeholders and social learning over time. Capacity building amongst stake-holders for the deployment of emergent methods to tap into collective intelligence will be required. New quality criteria and quality control mechanisms are re-quiredforknowledgeco-creationprocesseswithdiverse stakeholders[41,42].

willhelptostabilizenetworkbuildingforsociallearning to occurovertime.Suchresearchprojectswillprovidea demand for and anopportunity to betterleverage what thestudyprogrammehastoofferintermsofequipping scientists and citizens for engagement in sustainability scienceforsystemicchangeinLuxembourgandbeyond.

Acknowledgements

Nosinglemindcouldhavecreatedthisstudyprogramme.Withoutthehigh levelofengagementofcolleaguesattheuniversityfromallfaculties,the LuxembourgInstituteforScienceandTechnology,theMinistryfor SustainableDevelopmentandInfrastructuresofLuxembourg,STATEC, EUROSTAT,theEuropeanInvestmentFund,andnumerousother organizations,andallparticipantsthisprogrammecouldnothavebeenbuilt. DrJeromeRavetzprovidedintellectualandmoralsupportandcontinued substantialinputsthrough-outthebuildingoftheprogramme.The programme’sdevelopmentandimplementationwasfundedbythe UniversityofLuxembourg.

References

and

recommended

reading

Papersofparticularinterest,publishedwithintheperiodofreview, havebeenhighlightedas:

 ofspecialinterest ofoutstandinginterest

1. KuhnTS:TheStructureofScientificRevolutions.3rded..Chicago:

TheUniversityofChicagoPress;1962.

2. JerneckA,OlssonL,NessB,AnderbergS,BaierM,ClarkE, HicklerT,HornborgA,KronsellA,Lo¨vbrandE,PerssonJ: Structuringsustainabilityscience.SustainSci2011,6:69-82

http://dx.doi.org/10.1007/s11625-010-0117-x.

3. FeyerabendPK:Threedialoguesonknowledge. Oxford:

Blackwell;1991.

4. FuntowiczS,RavetzJR:Scienceforthepost-normalage.

Futures1993,25:739-7550016-3287/93/07739-17.

5. NowotnyH,ScottP,GibbonsM:Re-ThinkingScienceKnowledge

andthePublicinanageofUncertainty.PolityPress;2001.

6. KatesRW,ClarkWC,CorellR,HallJM,JaegerCC,LoweI, McCarthyJJ,SchellnhuberHJ,BertBolin,DicksonNMetal.: Sustainabilityscience.Science2001,292(4):641-642http://

dx.doi.org/10.1126/science.1059386Retrievedfromwww.

sciencemar.org.

7. MillerTR,WiekA,SarewitzD,RobinsonJ,OlssonL,KriebelD, LoorbachD:Thefutureofsustainabilityscience:a solutions-orientedresearchagenda.SustainSci2013,9:239-246http://

dx.doi.org/10.1007/s11625-013-0224-6.

8. Ko¨nigA:RegenerativeSustainableDevelopmentofUniversities

andCities.EdwardElgar;2013.

9. SchneidewindU,Singer-BrodowskiM:Transformative Wissenschaft:KlimawandelimdeutschenWissenschafts-und Hochschulsystem.Metropolis,Marburg;2014::419:.http://www.

nachhaltigewissenschaft.blog.de.

10. 

WalsAEJ,BrodyM,DillonJ,StevensonRB:Convergence

betweenscienceandenvironmentaleducation.Science2014,

344:583-584.

Acompellingcallforembracingcitizenscienceprojectstoostersocial learningonenvironmentalissues.

11. WiekA,LangDJ:Transformationalsustainabilityresearch

methodology.In SustainabilityScience—AnIntroduction.

EditedbyHeinrichsH,MartensP,MichelsenG.Springer;

2015:1-12.

12. PetersS,WalsAEJ:Learningandknowinginpursuitof

sustainability:conceptsandtoolsfortrans-disciplinary

environmentalresearch.In TradingZonesinEnvironmental

Education:CreatingTransdisciplinaryDialogue.EditedbyKrasny

M,DillonJ.PeterLang;2013:79-104.

13. 

JasanoffS:Technologiesofhumility:citizenparticipationin governingscience.Minerva2003,41:223-244http://dx.doi.org/

10.1023/A:1025557512320.

Thispaperpresentsasetofquestionsthatcanbeconsideredasessence for sense-making indiverse groups about decisionsand associated trade-offsforjointlydefiningandstrivingforprogress.Whilstithasbeen writtenalreadyovertenyearsago,Ihaveyettoencounteranotherpaper ofcomparablevalueinstructuringactivitiesandorganizingand facilitat-ingdiscussionswithstakeholders.

14. JasanoffS:Orderingknowledge,orderingsociety.In Statesof

Knowledge:TheCoproductionofScienceandSocialOrder.Edited

byJasanoffS. Routledge;2004:13-46.

15. 

BarthM:TowardsSystemicSocialLearningInImplementing

SustainabilityinHigherEducation:LearninginanAgeof

Transformation.LearninginanAgeofTransformation(Earthscan).

Routledge;2015.

Thebookexplorestheroleofuniversitiesinfosteringsystemic social learningfortransformativechange.Itprovidesacomprehensiveoverview onapproaches,barriersandchallengesaswellasstrategiesonhowto overcomethese.

16. KeenM,BrownVA,DyballR(Eds): Sociallearningin

Environmentalmanagement:Towardsasustainablefuture,. Earthscan;2005.

17. SteyartP,JigginsJ:Governanceofcomplexenvironmental

situationsthroughsociallearning:asynthesisofSLIM’s

lessonsforresearch,policyandpractice.EnvironSciPolicy

2007,10:575-586.

18. DeweyJ:Experienceandeducation..Touchstone(Simon&

Schuster1997#1938);1938::91.

19. JaeggiR:KritikvonLebensformen.Suhrkamp;2011.

20. 

Boix-MansillaV:Learningtosynthesize:thedevelopmentof

interdisciplinaryunderstanding.In TheOxfordHandbookof

Interdisciplinarity.EditedbyFrodemanR,ThompsonKleinJ,

MitchamC.OxfordUniversityPress;2010:288-309.

Thisbookchapterpresentsapragmaticframeworkforachievingcritical crosstalkbetweenscientistsactiveindisparatedisciplinaryfields.

21. Lotz-SisitkaH,RavenG:Learningthroughcases:adoptinga

nestedapproachtocase-studyworkintheGold-Fields

participatorycourseinitiative.EnvironEducRes2004,10:67-87.

22. SeelyBrownJ,CollinsA,DuguidP:Situatedcognitionandthe

cultureoflearning.EducRes1989,18(1):32-42.

23. KolbDA:ExperientialLearning:ExperienceastheSourceof

LearningandDevelopment.PrenticeHall;1983.

24. SterlingS:HigherEducation,sustainabilityandtheroleof

systemiclearning.In HigherEducationandtheChallengeof

Sustainability:Problematics,PromiseandPractice.Editedby

CorcoranPB,WalsAEJ.KluwerAcademicPublishers;

2004:47-70.

25. GoughS,ScottW:HigherEducationandSustainable

Development:ParadoxandPossibility.Routledge;2007.

26. VesterF:TheArtofInterconnectedThinking.MCBPublishing

House;2012.

27. ReedMS,FraserEDG,DougillAJ:Anadaptivelearningprocess fordevelopingandapplyingsustainabilityindicatorswithlocal communities.EcolEcon2006,59:406-418http://dx.doi.org/

10.1016/j.ecolecon.2005.11.008.

28. ManhartS:KalkulierteKrise.Za¨hlen,RechnenundMessenals

GrundlagenderModerneArch+Zeitschriftfu¨rArchitekturund

Sta¨dtebau.2012:17-21.

29. RadermacherW:Thereductionofcomplexitybymeansof

indicators—casestudiesintheenvironmentaldomain.In

Statistics,KnowledgeandPolicy:KeyIndicatorstoInformDecision

Making.EditedbyOECD. 2005:163-173.http://

www.sourceoecd.org/general/economics/9264009000.

30. DefournyJ,NyssensM:TheEMESapproachofsocial enterpriseinacomparativeperspective(No.WPno.12/03)(p. 28).Lie`ge,2012.

31. WalsAEJ,HeymannFV:Learningontheedge:exploringthe

sustainable living. In Educatingfor aCulture ofSocialand

EcologicalPeace.EditedbyWendenA. StateUniversityofNew

York Press; 2004:123-145.

32. WalsAE,BlewittJ:Third-wavesustainabilityinhigher

education:some(inter)nationaltrendsanddevelopments.In

SustainabilityEducation:PerspectivesandPracticeAcrossHigher

Education.EditedbyJonesP,SelbyD,SterlingS.Earthscan;

2010:55-75.

33. 

MedemaW,WalsA,AdamowskiJ:Multi-loopsociallearningfor sustainablelandandwatergovernance:towardsaresearch agendaonthepotentialofvirtuallearningplatforms.NJAS -WageningenJLifeSci2014,69:23-38http://dx.doi.org/10.1016/

j.njas.2014.03.003.

Acompellingpaperdescribingmeritsandlimitationsofvirtualspaces associatedwithcitizensscienceprojectsasscalableplatformsforsocial learning.

34. GubaL:FourthGenerationEvaluation.Sage;1989.

35. WiekA,WithycombeL,RedmanCL:Keycompetenciesin sustainability:areferenceframeworkforcademicprogram development.SustainSci2011,6:203-218http://dx.doi.org/

10.1007/s11625-011-0132-6.

36. 

WiekA,TalwarS,O’SheaM,RobinsonJ:Towarda

methodologicalschemeforcapturingsocietaleffectsof

participatorysustainabilityresearch.ResEval2014,

23:117-132.

Thispaperdevelopsaframeworkandethodologicalschemeforcapturing societaleffectsofparticipatorysustainabilityresearch.

37. SterlingS:Thesustainableuniversity:challengeandresponse.

In TheSustainableUniversity.ProgressandProspects.Editedby

SterlingS,MaxeyL,LunaH.Taylor&FrancisGroup;2014:17-50.

38. RyanA,CottonD:Timesofchange:shiftingpedagogyand

curriculaforfuturesustainability.In TheSustainableUniversity.

ProgressandProspects.EditedbySterlingH,MaxeyS,LunaL.

Routledge;2014:151-167.

39. EvansJ,JonesR,KarvonenA,MillardL,WendlerJ:Livinglabs

andco-production:Universitycampusesasplatformsfor

sustainabilityscience.CurrOpinEnvironSustain2015,16:1-6.

40. RosenbergD,TrencherG,PetersenJ:Studentsaschange

agentsinatown-widesustainabilitytransformation:the

OberlinProjectatOberlinCollege.CurrOpinEnvironSustain

2015,16:14-21.

41. FuntowiczSO,RavetzJR:Peerreviewandqualitycontrol.In

InternationalEncyclopediaoftheSocial&BehavioralSciences,Vol

17.EditedbyWrightJD.Oxford:Elsevier;2015:690-694.

42. StirlingA:Ageneralframeworkforanalysingdiversityin science,technologyandsociety.JRSocInterface/RSoc2007, 4:707-719http://dx.doi.org/10.1098/rsif.2007.0213.

43. RavetzJR:Post-NormalScienceandthecomplexityof transitionstowardssustainability.EcolComplexity2006,

3:275-284http://dx.doi.org/10.1016/j.ecocom.2007.02.001.

44. GeelsFW,SchotJ:Typologyofsociotechnicaltransition pathways.ResPolicy2007,36:399-417http://dx.doi.org/

10.1016/j.respol.2007.01.003.

45. OsterwalderA,PigneurY:BusinessModelGeneration:A

HandbookforVisionaries,GameChangersandChallengers.John

WileyandSons;2010::288.

46. StringerSwartR,RaskinP,RobinsonJ:Theproblemofthe future:sustainabilityscienceandscenarioanalysis.Global EnvironChange2004,14:137-146http://dx.doi.org/10.1016/

j.gloenvcha.2003.10.002.

47. ElahiS:Herebedragons...exploringthe‘‘unknownunknowns.’’. Futures2011,43:196-201http://dx.doi.org/10.1016/

j.futures.2010.10.008.

48. MuffK:TheCollaboratory:ACo-creativeStakeholderEngagement

ProcessforSolvingComplexProblems.GreenleafPublishing;

2014::300.

49. BrownT:Designthinking.HarvardBusinessRev2008,86:84-92 141.Retrievedfromhttp://www.ncbi.nlm.nih.gov/pubmed/

18605031.

50. SengeP,ScharmerCO,JaworskiJ,FlowersBS:Presence:Human

PurposeandtheFieldoftheFuture.CrownBusiness,Random