Performance evaluation of collaboration in the design process: Using interoperability measurement

(1)

Science Arts & Métiers (SAM)

is an open access repository that collects the work of Arts et Métiers Institute of

Technology researchers and makes it freely available over the web where possible.

This is an author-deposited version published in:

https://sam.ensam.eu

Handle ID: .

http://hdl.handle.net/10985/9757

To cite this version :

Amir PIRAYESH NEGHAB, Alain ETIENNE, Mathias KLEINER, Lionel ROUCOULES

-Performance evaluation of collaboration in the design process: Using interoperability

measurement - Computers in Industry - Vol. 72, p.14-26 - 2015

Any correspondence concerning this service should be sent to the repository

(2)

Performance

evaluation

of

collaboration

in

the

design

process:

Using

interoperability

measurement

Amir

Pirayesh

Neghab

a,

*

,

Alain

Etienne

a

,

Mathias

Kleiner

b

,

Lionel

Roucoules

b

a_LCFC_Laboratory,_Arts_et_Me´tier_ParisTech,_Metz_Campus,_France b

LSISLaboratory,ArtsetMe´tierParisTech,Aix-en-ProvenceCampus,France

1. Introduction

Giventhestudiesonproductdesignsuchas[1,2],thegoalofthe designprocessistodevelopaproductthatmeetscustomerneeds. Indeed,inthisstageoftheproductlife-cycle,thedata,characterizing customerneeds,mustbeprocessedinseveralartifactsinorderto obtainknowledgeconcerningtheproducttodesign[3],andthus createoneormorerepresentationsofthelatter[4]. Howard[5]

studiesthephasesandmilestonesofseveraldesignprocess,from theneedsexpressedtotheproductdeﬁnition.

In the current design processes,collaborationis an intrinsic characteristic, particularly in the context of multidisciplinary design [6,7]. In fact, the design process is moving towards collaborative design process, which requires and encourages collaborationsbetweendesignersfromthebeginningofproduct life cycle [8]. According to [9,10], there exist a plethora of collaborationdeﬁnitionsdescribingseveraltypesofinteractions. These interactions in the design process mainly occur when

designerscommunicatetheresultsoftheiractivitiesthatsupport productdataexchange[11,12]. AccordingtoChiu[11], commu-nicationsanddataexchangesareprerequisitesforcollaboration. In addition, the author did an experiment in an academic environmentthatindicatesthatthedesignershavespentalmost half of their time focusing on communication. It is alsostated thateffectivecommunicationiscriticaltotheprojectparticipants ofcollaborativedesign.

Itisobservedthatdesignersmayencounterproblemsintheir collaborationssuchastechnologicalproblemsindatatransmission orinunderstandingthetransmitteddata[13]. Accordingto[17], someoftheseproblems,particularlythedifferentinterpretations of data,are resultsfromthe useof this datain heterogeneous environments.Theseheterogeneitiesarebasedonthedifferences between computing environments, languages,techniques, tools

[14],anddatasources[15,16],indifferentareasofexpertise.Such heterogeneities are noted by [17], [18] and [19] as sources of problems in collaborations. The heterogeneity problems often cause irrelevant[20],inadequate, impreciseor ambiguous data

[21,20].Theseproblemsmightalsocausefailuresintheindividual tasksofdesigners[22]. Thisisthereasonswhyformorethana Keywords: Designprocess Collaboration Dataexchange Performanceanalysis Interoperability Processmodeling ABSTRACT

Adesignprocess,whetherforaproductorforaservice,iscomposedofalargenumberofactivities connectedbydataandinformationexchanges.Thequalityoftheseexchanges,calledinthispaper collaboration,requires the abilityto exchange useful, understandableand unambiguousdata and informationtothedifferentdesignersinvolved.Inthispaper,aglobalframeworkisfirstsetforprocess/ productperformancemanagement.Then,theresearchquestionfocusesonthedefinitionandevaluation oftheperformanceof collaborations,and byextension,ofthedesign processinitsentirety.This performanceevaluationrequiresthedefinitionofseveralkeyelementssuchasobjecttoevaluate,the performancecriteria,indicatorsandactionvariables.Inordertodefinetheobjectofevaluation,this paperreliesonaliteraturestudyoncollaborationresultinginanECOREmeta-modelofcollaborative processes. The collaboration performance measurement is for its part based on the concept of interoperability.Thismeasureestimatesthetechnicalandconceptualinteroperabilityofthedifferent pairwisecollaborations.Thepaperisconcludedbyproposingatooledmethodologyforcollaborations’ performanceevaluationincludingtwomainphases:processmodelingandinteroperability measure-ment.ToolingisprovidedthroughtheEclipseModelingFramework(EMF)usingits(meta-)model edition,constraintvalidationandmodelcomparisonfeatures.Theapplicabilityofthemethodologyis alsoillustratedusingacasestudyindesign.

* Correspondingauthor.Tel.:+33685231328..

(3)

decade,alargenumberofreferencesindicatetheneedforstudies on collaborations and the management of their performance

[23–25],Despitetheseindications,veryfewtheoriesexplainhow to manage suchperformance particularlyin a situation where informationplaysakeyrole[26].

Thewordperformancecanbeappliedindifferentterms,such as‘‘performance management’’,‘‘evaluation ormeasurement of performance’’ and ‘‘performance assessment’’ to describe the processofcapturingperformance[27]. Inthispaper,theresearch approachisbuiltaroundthekeyelementsthatformaproblemof performanceevaluation.Basedon[28],itisconsidered thatthe evaluation of theperformance of an object (e.g.,a company,a designprocessora product)comprises thefollowing elements: criteria, indicators, action and measurement variables. Among these elements, the criterion choice plays a significant role sinceitreflectstheobjectiveoftheevaluation.Theperformance assessmentcanbebasedon classicalcriteriasuchascost,time and quality; in this paper, considering the significance of collaboration,itisdecidedtoretainacriterionthatintrinsically belongs to this concept and addresses the quality aspect of collaboration(seeFig.1).ThiscriterioniscalledInteroperability. Severalclassificationsandevaluationapproachesof collabora-tionwerestudiedtoidentifysuchcriteria.HustedandMichailova

[29]classifycollaborationas:infantile,repeatedandmaturebased oncollaboration history,wheresomeclassificationsaddressthe collaborationresults. In theseclassifications, thecollaborations canbesuccessful,significantandstrongorfailure,non-significant andweak[30–33]. Thethirdcategoryofclassificationprovides one or more criteria based on the difficulties encountered by collaborativeactors.Forexample,Girard[34]classifies collabora-tion as: free, encouraged or forced based on the freedom of actorstocollaborate.Indeed,inthisclassification,organizational difficulties(i.e.,authorizationsandmotivations)areunderlined.It is observedthat theclassifications andassessment approaches, suchas[35,36],whicharebasedonacriterioncalled ‘‘Interopera-bility’’,aremoresuitedtoevaluatethecollaborationperformance. Infact,thisconceptcoversalargerspectrumofcapacitiesrequired in collaboration.Regarding thedefinitionsproposed by [37,38], thegoalofinteroperabilityistoovercometheproblemsindata exchanges[22].

Inthispaper,inordertoclarifythisconceptwiththeobjective ofﬁndingatleastoneofitsindicators,theChen’sframeworkis adopted[39]. Thisframework,asillustratedinFig.2,structures interoperability in three dimensions: Interoperability barriers, concernsandapproaches.

ConcerningtheﬁrsttwodimensionsofChen’sframework,this paper studies the technological and conceptual barriers of interactions ofthetype dataexchange between designers. This paperalsoaddressesthethirddimensionsinceany interoperabili-tyapproachorsolution,appliedinacollaborativeprocess,should beidentiﬁedandstudiedintheevaluationofthecollaborations’ performance.ThisscopeislocatedinChen’sframework(seeFig.2). Considering this scope, the present paper focuses only on the problemsofdataadequacy. Accordingto[26,40]suchproblems areinexpansion.Inthispaper,itisprimarilyconsideredthatthe dataare always producedcorrectly and are understood by the

designers. This assumption allows to better focus on the inadequate data since this problem is dissociated from the accuracyandcomprehensibilityproblems.Dataadequacyisthen consideredasanindicatorofinteroperability.Hereitisconsidered that this indicator reaches its maximum value when the data produced by the actors: (1) can be accessed by others (this conditioncorrespondstothetechnicalaspectofthisindicator),and (2) are sufﬁcient and useful for the others. This condition correspondstothesemanticaspectofthisindicator.

The design process in this paper is considered to be in its evaluation and validation phase. Therefore, the objective is to evaluatethisprocessbefore itsexecution.Inthis case,onlythe backboneoftheprocess,includingthesequenceofactivitiesand resources,isavailable.Insuchprocess,productdataarenotyet instantiated.However,someinformationisavailableonthem:file exchange formats and file abstractions (the file contents and structure of data) in case of availability of these abstractions. Theextractionoftheabstractionisusuallyatediousorimpossible task,especiallywhentheformatisnotopenorisnotbasedonan openstandard.Therefore,thispaperislimitedtothetermsusedfor fileformatswhentheabstractionsarenotavailable.Havingthis information on the data files, data adequacy conditions are translatedastheCompatibilityoffileformatsandcontents.

Considering the fact that in this papercollaboration perfor-manceisdefinedastheabilityofactorstoexchangeadequatedata, thefollowingquestionsareraisedhere:isitpossibletoquantify and improvethe collaborationperformance? How thelatter is affected by the elements of collaboration (actors and their resources)orbytheirheterogeneities?Facingthesequestions,it is first considered that collaboration performance, as a part of processperformance,impactsproductperformance.Suchimpacts arementionedpreviouslyinthefieldofriskanalysis,particularly in [41,42],which study thelinks between processand product performancesbasedonproductfailures.

AsillustratedinFig.3,theperformanceevaluationbasedon Interoperability (i.e., 1a) and theparallel evaluations based on othercriteria (i.e.,1b) canbepositionedin a globalframework supportingthemanagementofthedesignprocess.Thisframework is founded on thelink between the resultsof the process and productperformancesthroughproductfailureanalysis(i.e.,2).This analysis shows that all the results of process performance evaluationcanbeanalyzedbyfocusingontheproblemscausing product failures. In the generating step (i.e., 3), alternative processes can be generated by modifying the elements of collaborations identiﬁedas criticalin theprevious step.During theselectionstep(denoted4),anoptimizationmodelisusedto maximize thevalue of theproduct. Consideringthe Fig.3 and Collaboration

performance

Quality

“Interoperability”

Fig.1.Interoperabilityasthecriterionofcollaborationperformanceevaluation.

Barriers Technological Conceptual Organizational

C o nc e rns Business Process Service Data Process Service Data Service Data Data Scope of this paper Integration Federation Unification

(4)

facingtheresearchquestions,inthispaperthepropositionforstep 1a of this process is discussed. This proposition is a tooled methodologyforperformanceevaluationofcollaborationbasedon dataadequacyasoneindicatorofinteroperability.

2. Literaturereview

Thissectionpresentsatﬁrsttheliteratureofinteroperability measurement.Theninteroperabilitysolutionsarediscussed. 2.1. Interoperabilitymeasurement

Aconsiderableamountofworkhasbeendoneontheneedfor interoperability inside and outside organizations. A statistical studyofthisconceptindicatesthatithasbeenusedasakeywordin theliteraturefornearly50 years[43]. KasunicandAnderson[44]

believethatinteroperabilityisabroadandcomplextopicandthe development of speciﬁc measures and their application is a difﬁcult task. However, these authors recognize that assessing interoperabilitywithwell-chosenmeasuresisessentialtoidentify priorities to consider within business collaboration. Many researchershavebeeninterestedinsuchassessmentsandmany approaches are proposed in the literature. Some of them are studiedin[36].Theseapproachescanbestudiedintwocategories: evaluationoftheinteroperabilitybasedon independentcriteria such as cost, time or quality and evaluation of the level of realizationofinteroperabilitybasedoncriteriamoredependingto thisconceptsuchasdegreeofcouplingorcompatibilityandthe onesevaluating. Approachesproposedby [45–47]arebased on cost, time or quality. Such approaches evaluate directly these criteriainenvironmentsrequiringinteroperabilityortheimpactof interoperabilitybarriersonthesecriteria.Forinstance,Chenet al.

[47]proposeanapproachforquantitativeevaluationofthecost, time and quality in aninteroperation process. In theapproach proposed by[35,47], theinteroperability potentialis evaluated throughasetofprovisionsthatmayimpacttheinteroperability suchastheflexibilityandopennessoftheactorsofcollaboration. Theseprovisionscanbeassessedtheinteroperabilityofanobject without prior knowledge of its collaborative partners. This assessmentidentifies the riskof encountering problemsduring theestablishmentofcollaboration.Fourcriteriaareappliedinthis assessment:openness,coupling,centralizationand configurabil-ity. Eachcriterion hastwo quantifiable levels.For example,an objectcanbeassessedasopenorclose,basedonthefirstcriterion. Other authors consider that the coupling degree between collaborativeobjects(actorsandsystems)indicatetheir interoper-ability.Bianchini[48]proposesanapproachthatfirstlyexpresses thesemanticrelationshipsbetweentheseobjectssinceeachobject ismodeledasacollectionofserviceprovidingprocesses.Thus,the interoperabilityevaluation is performedbasedon the degree of couplingbetweentheseprocesses.Intheapproachproposedby[49], the degree of coupling is measured by extent and intensity of information indicators. Actors are considered not coupled or

stronglycoupledwhentheseindicatorsarerespectively0or1. In the ﬁrst case no information exchange between actors of collaborationarerealizedandinthesecondcase,allinformation areavailabletobesharedandusedbyalltheseactors.

DucqandChen[50]considerthecompatibilitymeasurementin their interoperability evaluation approach. In this approach, a questionnaireisappliedwhichtakesintoaccountinteroperability barriersandconcernsintroducedbeforeinsection1. Theresults ofthisquestionnairearelinkedintoatableincludingthevalues of compatibilitymeasurement for each pairof barrier-concern: 1whennonincompatibilityisdetectedand0 otherwise.Ford[51]

proposea methodology in twostepsto quantitativelymeasure the interoperability. The first step is called Measuring I-score (Interoperability-score)throughthequantificationoftheintrinsic property of a given network called interoperability spin. This propertycanqualifythevalueofinteroperationbetweenpairof objects:1 wheninteroperabilitycanbeachievedwithoutmanor machineinterventionforinformationtransfer,0and1 respec-tivelyincaseofmachineorhumanintervention.Thesecondstepis themeasurementofoptimumI-scoreifmodificationsarepossible whichaffecttheinteroperabilitybarriers.

Some of the interoperability measurement approaches are basedonmaturitymodelsthatevaluatethestagesanobjectmust followtoachievearequiredlevelofperformancefor interopera-bility.Severalmaturitymodelshavebeendeveloped:Spectrumof InteroperabilityModel(SoIM)[52],TheQuantiﬁcationof Interop-erability Methodology(QoIM) [53], MilitaryCommunications & Information Systems Interoperability (MCISI) [54], Levels of InformationSystemInteroperabilityModel(LISI)[55], Interopera-bilityAssessmentModel(IAM)[56],Organizational Interoperabili-tyModel(OIM)[57],LevelsofConceptualInteroperabilityModel (LCIM) [58], EnterpriseInteroperabilityMaturity Model(EIMM)

[59],ISO11354-1:2010[60]. Thesematuritymodelsmayallow assessmentofpotentialinteroperabilityortheabilityto interop-erate with others. Some of these models are studied and characterizedby[61].

Yahiaet al.[36]studyinteroperabilityindataexchangesonly based on conceptualbarriers. In this approach, a method called semantic matching plays a specific role particularly for the measurementofcompatibilities.Accordingto[15],similarmethods areapplied inthefollowing areas: ontologyintegration,message transformation between databases or data directories and more recentlyin thecoordinationofweb services[15]. As mentioned above,theultimategoalofmatchingistofindthedifferencesbetween twoobjects. These differencesarenamed in severalways inthe literature:dissimilarity[15],mismatch[62]orconflict[19].Different methodsofmatchingareproposedintheliterature[63]. Detailed literaturereviewarecarriedoutin[64]. Concerningtechnologiesor tools,whichhavetheobjectiveoffacilitatingthematchingprocess, Luconductedastudyonthetechnologiesappliedincertaincategories ofmatching,especiallyonthebasisofautomationofthesimilarity measure[65]. Manyofthesetechnologieshavealsobeenstudied in[66,67].

(5)

Although matching is widely applied as compatibility mea-surementapproach,thechoiceofitsmethod, operatorandtool depends mainly on the objectives of matching, objects to be studiedandtheircharacteristics[84].Indesignprocessesincluding datamodelingapproachessuchasModelDrivenEngineering,the matchingisessentialforthemanagementofmodels[68]. Infact, when the models are large and complex (containing a large numberofentities),dedicatedtoolsareneededtocomparethem

[86]. Tothis end,modelcomparisontechniquesaredeveloped. Stephan and Cordy [69] discuss model-to-model comparison techniques.Theyillustrateandevaluatequalitativelycomparison techniques, whilehighlightingthestrengthsandweaknessesof each the studied techniques. These authors show that EMF Compare, which is applied in this paper, has the best overall performance compared with techniques such as ECL, C-Saw, DSMDiff,andSmoVer.However,EMFCompare,ECL,andSmoVer only work withEclipse Modeling Framework (EMF), but other techniquescanbeextendedtoworkwithotherframeworksand modelinglanguages[69].‘‘EMFisamodelingframeworkandcode generationfacilityforbuildingtoolsandotherapplicationsbased onastructureddatamodel’’[70]. Anexampleofapplicationof EMF compare can be found in [71], which introduces a new approachforcomparingmodelsusingthistool.Thisapproachis basedonacomprehensivecomparisonofmodelcomponents.The author proposesalso a systemof weights which calculatesthe magnitudeofthedifferencebetweentwoelementsofamodel[71].

2.2. Interoperabilitysolutions

In interoperability measurement, the actions available that canimprovethemeasuredvaluemustbealsoidentiﬁed.These action plans can be interoperability solutions embraced by the approaches of interoperability, the third dimension of the frameworkofChenpresentedearlierintheintroductionsection. Concerningthedevelopment andapplicationof thesesolutions, companies may have different modeling strategies. Indeed, modeling of the applied domains, particularly in the design context,canbebasedonauniquemodelfortheprocessorbased ontheapplicationofseveralbusinessmodels(seeFig.4).These models,standardorbasedonanagreementwithinthecompany, focus on ﬁle exchangeformats, modeling languages and meta-models (ontology). In the case of using multiple models in collaborations, complementary interoperability solutions such as model transformation is required to resolve the issue of heterogeneousmodels.

2.2.1. Standardmodels:dataﬁleformats,modelinglanguagesand ontology

This solution focuses on the use of standards models and formats(e.g.,STEP)indataexchanges[72–74]conductedsurveys onthestandardsusedinthelifecycleofproducts.In [73],itis ﬁrstlynotedthattheexchangeofinformation,betweenproducers andconsumersofinformation,requiresalanguagethatconveythe contentoftheinformation.Thislanguageisdeﬁnedmainlybythe type of this content, its structure and thedata instances [73].

AccordingtoTerzi[74],manystandardscoverproduct develop-ment,production anduse.Currently, severalCAD/CAM applica-tionscontainamoduletoreadandwritedatadeﬁnedbyoneofthe STEPApplicationProtocols[75]. Thelackofﬂexibility,dynamism and automation of standards and the increasing volume of productdatahaveledtothedevelopmentofontology-basedon approaches[17].

Regarding standard models, ontology and meta-models can alsobementioned.Accordingto[76],aproductontology,obeying the rules of theﬁeld, provides theopportunity toexpress and share product data between: actors, business applications and informationsystems.Furthermore,inacollaborativeenvironment, ontology play a vital role in the interoperability of different systemsinaparticularareabyemphasizingonthesemanticaspect ofdataintegration[77].

In the context of product development, domain ontology usually offers a description of areas: simple, comprehensive, understandableandimplementablebyhuman.Arecentsurveyof domainontologyispresentedby[17]. Thissurvey,mainlyabout the use of inference-oriented ontology (reasoning about data), studiestheontologyinthreedimensions:stagesofthelifecycleof products,thescaleoftheproposedmodelsandﬁnallytheinvolved ﬁelds(product,processorservice)[17]. Ontologycanalsobeused in annotation to overcome semantic interoperability barriers. According to [78], annotation is a factor that enhances the semantic interoperability as it promotes the exchange of data between collaborative actors.In fact,these actorscanannotate theexchangeddatatoenrichthesemanticsoftheirentities. 2.2.2. ModelTransformation

Usingasinglestandardorreferencemodelasinteroperability solutionrequiresaneffortbytheuserstounderstandtheconcepts andthemethodsofimplementation.Therefore,thestandardmust beadaptedtoallstakeholdersina consensualmanner,since in some areas, it is almost impossible. Currently, considering the diversityofactorsintheproductdevelopment,severalstandards or models are used in the data exchange. In this case, some treatmentsonmodelssuchasthemodeltransformationshouldbe considered inordertoachieveinteroperability.Model Transfor-mationisdeﬁnedastheprocessofconvertingasystem[product] model to another model of the latter [78,79]. Czarnecki et al.

[80] propose a taxonomy of classiﬁcation of different existing approachesformodeltransformation.Thistaxonomyisdescribed by a functionality model, which explicitly supports choices of model transformation development. The authors also classify existing approaches as follows: direct manipulation, relational, graph-based, and structure-driven and hybrid. The mechanism ofmodeltransformationismoredetailedheresinceitisapplied inthecasedstudyofthispaper.Forthispurpose,thearchitecture illustrated in Fig. 5 is applied which is based on Meta Object Facility[81].

(6)

Inthisarchitecture,themainobjectiveistotransformasource modeltoatargetmodel.Thesemodels,atM1level,conformto theirownabstractions,themeta-modelsatM2level,andthese abstractionsconformtoasinglereferencemodel,the meta-meta-modelatM3level.Regardingcollaborativeprocessesindesign,the sourcemodelcontainsproductdatathatareprovidedbyadesign activityandthetargetmodelisbasedonthedatarequiredbyother activities.Thesemodelsconformtotheirownmeta-modelsor,in otherwords,totheir modelinglanguages.Thetransformationis basedontransformationrules:mapping.Millerdeﬁnesthelatter asthespeciﬁcationofamechanismfortransformingtheelements ofamodel,conformingtoameta-model,particularlyintoelements ofanothermodel,conformingtoanothermeta-model[79]. The transformation also requires a transformation language that implementsthe transformation rules. This language itself may conformtoM3levelinarchitectureoftransformation.

3. Methodology

Consideringtheglobalframeworkpresentedinthe introduc-tionsection,theproposedmethodologyaddressestheﬁrststepof thisframework(Step1ainFig.3).Enterprises,developingproducts throughmultipledataexchangesintheirprocesses,canapplysuch methodology.Themethodologyiscomposedoftwomajorphases (illustratedbytheIDEF0diagraminFig.6):

Phase1a.1,processmodeling:thisphaseincludesmodelingthe designprocessbyputtinganemphasisoncollaborationsandonpair ofdataexchangesbetweendesigners.Inthisphase,theﬁrsttaskis theprocessmodelinstantiation. In the secondtask, the process modeliscompletedaccordingtoprocessexpertknowledge.

Phase1a.2,interoperabilitymeasurement:thisphaseconsists of a measure of performance basedon interoperability.In this phase, according to the information provided by the previous phase (1a.1),the valueof an indicatorof interoperability,Data Adequacy,ismeasured.Intheﬁrsttaskofthisphase,thesyntax matchinginthepairwise dataexchangesareveriﬁed.Then the second task is performed in order to quantify the semantic matchingbycomparingdataabstractions.

In thefollowing,thetooling environmentisﬁrst introduced. Then,thekeyelementsofthismethodologyaredetailed. 3.1. Toolingenvironment

Inordertosupportthemethodologyproposedinthispaper,itis suggestedtouseasetoftoolsprovidedbytheEclipseModeling

Framework (EMF), chosen as the implementation framework becauseofitsmaturityandlargetoolsupport.

The different chosen tools and their use in each phase is illustrated in Fig. 6 (as bold italic). These tools are brieﬂy introducedbelow.Theirconcreteapplicationinourmethodology isdetailedinthefollowingsectionsalongwithitsdifferentsteps. ECORE[82]is ameta-meta-model,basedon OMG’sMOF2.0

[83],whichimplementsitssubsetEssential-MOF(EMOF).ECORE allowsdefiningdomainspecificlanguages(inourcase,theprocess meta-model).AlltheotherEMFtoolsactonECOREmeta-models. EMFprovidesadefaultmodeleditor(inourcase,toinstantiatea givenprocessmodel),andoffersthepossibilitytodefinecustom editors,includinggraphicalones(usingforinstanceGMF). EMF also offers a set of validation tools, built around the Object ConstraintLanguage(OCL)[83],allowingtofurther specifyand checkvariouspropertiesoncreatedmodels.Finally,EMFcompare is a toolfor analyzingthedifferences between(meta-) models, whichinourcaseisusedinthesemanticmatchingphase. 3.2. Meta-modelofcollaboration

Thismeta-modelisthecoreofphase1a.1,whichaimstoanalyze the collaborationperformance.Thecollaborationmustbefirstly defined by emphasizing on its key elements. Concerning this definition, a limit was initially expressed by [10]: a clear and conventionaldefinitionofcollaborationisdifficulttoachieve.Facing theseissues,severaldefinitionsofcollaborationswerestudiedand analyzed.Asanexample,thefollowingdefinitioncanbementioned which is proposed in the field of collaborative design [34]: ‘‘collaborationisdefinedaccordingtotwoaspects.Thefirstconcerns thefreedomofdesignerstocollaboratewitheachother.Thesecond isareflectionofthedesignteam’scollaborativeexperience.’’The extracted definitions from literature were analyzed using text-miningtools:TextContentAnalysisTool[139]andKHCoder[140]. Theseanalysesledtoidentificationofkeyelementsofcollaboration. Theseelementsaremodeledasaconceptualmodelofcollaborative process. The conceptual model is also extended based on the elements introduced by the use of interoperability solutionsin thedesignprocesses.Eventually,thisconceptualmodelis formal-ized as an ECOREmeta-model, represented in Fig. 7 as a class diagram,givingourmethodologyaccesstoEMF’smodeleditionand validationtoolsforprocessinstantiations.AsetofOCLconstraints furtherspecifythevalidityoftheinstantiatedprocesses.

Belowisabriefexplanationofthemainconceptsofthis meta-model:

(7)

‘‘CollaborativeProcess’’,representingthewholedesignprocess,is therootofthemeta-model.

‘‘DesignActivity’’representsthedesignactivitiesintheprocess withasingledesigner.

‘‘CollaborativeActor’’modelsthedesigners workingon design activitieswiththeirsoftwaretools.

‘‘ProductDataFile’’ represents the exchanged files containing theproductdata. Eachfile hasa status:produced,delivered or required. Indeed, a file can be produced, required by a design activityoritcanbedeliveredtoanactivitybycommunication.Each datafilehasalsoafileformat.

‘‘AbstractionOfDataFile’’ represents the data structure of its associateddataﬁle.

‘‘CommunicationType’’representsaparticularmodeof commu-nicationavailableintheprocess.Itcanbeasolution,whichaffects thesyntaxorsemanticsofadataﬁlesuchasmodel transforma-tions. In fact, a Communication Type is not a design activity parameterbutitsupportstheinteroperability.Itcanbeappliedin oneorseveralcommunications.

‘‘Communication’’modelstheunilateralandunidirectionaldata ﬂowfromoneactivitytodownstreamone.Eachcommunication hasasequencethatdesignatesthechronologicalcombinationof CommunicationTypesappliedinthecommunicationprocess.

‘‘CollaborativeWork’’ is an intermediate class that embraces twodesignactivitiesassociatedwithauniquecommunication.In reality,thisclassisinstantiatedbytheexistenceoftheseelements. 3.3. Granularityofthemeasurement,pairwisecollaboration

Intheproposedmethodology,acollaborativeprocessisstudied by focusing on its sub-elements that meet still respect the

deﬁnitionofcollaboration.Indeed,acollaborativeprocesscanbe decomposedintominimalelements whichthemselvesestablish collaboration. This minimal unity, called Pairwise Collaboration (PwC), is formed by a communication between two activities. Indeed,aPwC,illustratedinFig.6,it-selfisacollaborationthat mainlycontains:

Twoactorswiththeirownattributes,activitiesandobjectives, butwithasharedobjectiverealizedthroughcollaboration,

two activities, one producing structured data and one consumingtheproduceddata,

acommunication,singleandunilateraldataexchange, receiv-inganddeliveringproduceddata.

3.4. Performancemeasurementofapairwisecollaboration

As explained in Section3.1, the communication element of collaborationmayincludeotherelementscalled Communication-Typesthatcanmodifydatabeforetheirdelivery. Therefore,it is considered that the measurement of the collaboration perfor-manceshouldbeperformedintwo[vertical]levels(Fig.8):

Potentiallevel:Thislevelindicatesthecompatibilitybetween producedandrequireddata.

Currentlevel:Thislevelindicates thecompatibilitybetween deliveredandrequiredﬁles.

In order to measure the Data Adequacy in each level, two matching methods are used. In these matching, formats and abstractionsofdataﬁlesarethemeasurementvariables.Having thevalueofthesevariables fromtheprocessmodel,thesyntax and semantic matching are performed respectively associated withformats(A,BandCinFig.8)andtheabstractions(a,bandc inFig.8).

(8)

(A)Syntaxmatching

For thesyntax matching of file formats, a simple similarity function(basedonterminologicalsimilarityofterms)isused.This functionverifiestheterminologicalequivalenceoffileformats.For example,havingtwofilesFD(delivered)andFR(required),with

respectiveformatsBandC(seeFig.8): B¼C!SimilarityfunctionðFD;FRÞ

¼True&B6¼C!SimilarityfunctionðFD;FRÞ¼False (1) Based on these similarityfunctions, for allcollaboration the matching rules indicated in Table 1 must be veriﬁed. Two mechanisms, manual and automatic, are proposed for this purpose:

Manualsyntaxmatching:Therulesofsyntaxmatchingcanbe verifiedmanually.To this end,theprocessmodel mustfirstbe studiedforeach collaboration(representedby theCollaborative Workclass)inordertoidentifyexchangedfiles.Then,theformats ofthesefilesarecomparedbasedonmatchingrules.Themanual syntaxmatchingcanbetediousandcomplexforcaseswithseveral collaborations. Therefore, an automatic mechanism is also proposed.

Automated syntax matching: the use of OCL constraints is proposed to increase the automation of matching. These con-straints are added tothe meta-model of process based on the matchingrules.Inthismeta-model,theclass‘‘CollaborativeWork’’ isthecontextoftheOCLconstraints.

In order to increase the precision of data compatibility measurement, the matching can be done in a second level, semanticmatching,incaseofhavingopenformatsandavailability ofthedatastructures.

(B)Semanticmatching

Inthemethodology,thesemanticmatchingisdeﬁnedbasedon thedata compatibilityat aconceptual(abstract) level.Here,an abstractionCisdeﬁnedasathreelevelconceptualstructure: AbstractionC¼fc1;...cng (2)

where,ciisaclassinCand

ci¼fcpi1;:::;cping (3) where,cpijisapropertyoftheclassci.Theﬁrstpropertyofthe

classiscpi1isunique(themandatoryidentiﬁerwhichismainlythe

classtitle)butotherpropertiesareoptional;theymaychangefrom oneconcepttoanother.Anoptionalpropertycanbeindependent, asanattribute,orrelatedtoanotherclass,asanassociation.Here, theheritage(generalization)associationisnotconsidered asan association.Meanwhile,eachclasshavingsuchrelationinherits theoptionalpropertiesofitsupperclass.Theoptionalproperties canalsohavetheirownobligatoryandoptionalproperties: cpij¼fcppij1;:::;cppijmg (4)

These properties are limited based on the formalism of abstraction.For example,in thispaperforaUMLclassdiagram oranEMFmeta-model,threepropertiesareconsidered:

Propertiesofattributes:type,lowerandupperbounds, Propertiesofassociations:type(aggregationorcomposition), minormaxcardinalities,

Intheproposedprocessofsemanticmatching,theabstractions mustfirst beadaptedtotheabovedefinitionofanabstraction. Once this condition is verified, a similarity function based on textual equivalence of theentities ofabstractions (concepts)is used for semantic matching. Indeed, it is considered that equivalenttermsrepresenta uniqueconceptfordesigners. This assumptionisduetoanissuelocatedmainlyoutsidetheperimeter ofthispaper(itcanbestudiedinlinguistics).However,incaseof availability of domain ontology in the design process, this assumptionismorerealistic.

HavingtwoabstractionsXSourceandXTargetbasedonthesame

concrete syntax, the similarity function veriﬁes if the source includesalltheconceptsidentiﬁedasrequiredinthetarget: SimilarityfunctionðXSource;XTargetÞ

¼True!XSourceXTargetð8c2XTarget:

9

c02XSource:c0¼cÞ

(5) Theresultsof thesimilarity functioncan showmismatches, missing elements of the target in the source. Based on the conceptuallevelsofanabstractionamismatchcanbe:

Mismatchbetweentheconceptsoftheﬁrstlevel(classes):This typeofmismatchoccurswhenaclassofXTargetdoesnotexistinthe

XSource.

Mismatchbetweentheconcepts ofthesecondlevel (proper-ties): This type of mismatch occurs when for a class of XTarget

Fig.8.Differentlevelsofperformanceevaluationinapairwisecollaboration.

Table1

Syntaxmatchingrules. Rule Matchinglevel

1 Potential:Incollaboration,the‘‘produceddata’’withﬁleformat‘‘A’’shouldhavethesameﬁleformat‘‘C’’asthe‘‘requireddata’’.!Similarityfunction:A=C OCLconstraint1(context:CollaborativeWork):self.hasAvProducing.avProduces.format=self.hasAvRequiring.avRequires.format.

2 Current:Incollaboration,the‘‘delivereddata’’withﬁleformat‘‘B’’shouldhavethesameﬁleformat‘‘C’’asthe‘‘requireddata’’!Similarityfunction:B=C OCLconstraint2(context:CollaborativeWork):self.hasAvProducing.avProduces.format=self.hasAvRequiring.avRequires.format;

(9)

alreadyexistsininXSourcebutaproperty;attributeorassociation,

ofthisclassismissinginXSource.

Mismatchbetweentheconceptsofthethirdlevel(propertiesof properties):Thistypeofmismatchoccurs:whenaclassofXTarget

and its attribute/association exist in XSource, but a property of

this attribute/association, for example the lower cardinality is different.

Themismatchescan beweightedaccordingtotheir relative importance. This weighting can be based on the opinion of theexpertprocessoractorsofcollaboration.Inthispaper,these weightsarerespectivelysetto1,0.5 and0.1.

Aftertheidentiﬁcationofmismatches,thesemanticmatching valueisestimatedbythisequation:

SemanticMatchingValue¼1WME

WE (6)

where,WEandWMEarerespectivelytheweightedvaluesof theelementsoftheXTargetandtheweightedvalueofmismatches

(missing element of XTarget). The parameters of Eq. (6) are

calculatedasbelow:

WE¼NCWCþNPWPþNPPWPP (7) WME¼NMCWCþNMPWPþNMPWPP (8)

where,

NC:Numberofconceptsoflevelone(classes)inXTarget,

NP:Numberofconceptsofleveltwo(optionalproperties)in XTarget,

NPP:Numberofconceptsoflevelthree(optionalpropertiesof properties) in XTarget, as explained before in the simpliﬁed

deﬁnitionofanabstraction, hereonly threeoptional properties areconsideredforeach property.Therefore,thevalueofNPPis threetimesthevalueofNP.

WC,WPandWPP:weightsoftheconceptinlevel1,2and3. Theseweights mustdeﬁned bythedesigner receivingthedata basedontherelativeimportanceofconceptsateachlevel.

NMC,NMP and NMPP:Number ofmismatches between the conceptsoflevel1,2 and3.

The calculated value of semantic compatibility can be comparedtoathresholddefinedbasedontheopinionofactors of collaboration. This threshold indicates the amount of data considered as sufficient for these actors. Here, the minimum thresholdissetto0.5. Forallcollaboration,semanticmatching rulesincludedinTable 2mustbethereforeverified.

The semantic matching as well as the syntax matching is performedattwolevelsincollaborationandresultsmayindicate a currentora potentialvalueof theperformance.Thesameas syntaxmatching,semanticmatchingcanbeperformedmanually orinanautomaticway:

Manualsemanticmatching:Therulesofmatchingsemantics canbeverifiedmanually.Tothisend,theprocessmodelmustfirst bestudiedfor each collaboration(represented by the Collabor-ativeWorkclass)in ordertoidentifyfilesexchanged. Then, the abstractionsofthesefilesareusedintheverificationofmatching

rules.Themanualmatchingcanbetediousandcomplexcaseswith abstractions withseveral concepts of in case of havingseveral collaborationsintheprocess.Therefore,anautomaticmechanism toachievethismatchingisalsoproposed.

Automatedsemanticmatching:inordertoincreaseautomation inthecomparisonofdatamodels(basedonthesemanticsofthe design domain), it is proposed to useEMF Compare. Thistool providesautomaticmatchingbetweentwoECOREmodelsbased on structural and textual comparisons. However since these comparisonsmaysometimesbeinaccurateregardingthe seman-tics of the design domain, the given results should only be considered hints and require further veriﬁcationsby a design/ processexpert.

4. Casestudy

Inordertoverifytheapplicabilityoftheproposed methodolo-gy, a case study is performed. The latter is based on a design processofamechanicalcouplingbetweenapropellerandadiesel engine. The general information on this case is ﬁrst presented. Then, the application of themethodology and obtained results arediscussed.Itshouldbementionedthattheapplicationofthe methodologyinthiscasestudyiswithahigherobjectiveforits user(theprocessmanagement).Thisobjectiveisthecomparison and analysis of design process and its alternatives in order to select the best one based on the results of interoperability measurement.Thecasefocusesonapartofthisprocessillustrated inFig.9.

AsillustratedinFig.9,auniquePairwiseCollaboration(PwC)is analyzed.Theobjectiveofthiscollaborationistoexchangedataon theproductcomponentsandtheirenergyflows,basedonamodel calledSK2[84],inordertofacilitatethegeometricaldesign.Inthis collaboration,thefirstandsecondactivityrespectivelyproduces andrequiresdata.Therefore,thefocusisontheresultsofthefirst activityand thedatarequired bythesecondone. Therequired datamustbeinSTEPAP-203format,which isconsideredasan exchangeformatforgeometricaldata.Forthecommunicationpart ofthiscollaboration,analternativeprocessexistsintheprocess. InFig.9,thealternativeprocessisaprototypethathasbeen developedbasedonafederativeapproachin ordertoverifythe possibilityofreplacingdesignactivitiesbymodeltransformations. Therefore,twoconsecutivemodeltransformationsareappliedin thecollaboration.ThefirstonetransformsaSK2modelintoand intermediate model calledASB; thesecondone transformsthe latterintoan STEP-AP203model.Moredetailed informationon thesetransformationarefoundin[84]. Itshouldbenotedthat, aftertheapplicationofthesetransformationsin thealternative, not only the communication but also some other elements of theinitialcaserequiremodifications.Forexample,theformatof DataFileDF06_Dwillbebasedon‘‘STEPXML’’insteadof‘‘MSVisio Drawing’’sincetheoutputformatofthemodeltransformationis basedonthefirstone.

4.1. Methodologyapplication

Havingthenecessaryinformationonthedesignprocessandas wellastheopinionoftheexpertprocess,themethodologycan beapplied.

Instantiation of the process model: For phase 1a.1 of the methodology (process modeling), a speciﬁc process model is instantiatedfromourmeta-modelofcollaborationpresentedin Section3.2. Theprocessexpertprovidestheinformationinput. Technically, this is realized using EMF’s model editors, which usesXMIasaserializationandexchangeformat.Anexcerptofthe tree-view ofthegenerated model forthecase initialversion is showninFig.10.

Table2

Semanticmatchingrules. Rule Matchinglevel

1 Potential:Incollaboration,theabstraction‘‘a’’ofthe ‘‘produceddata’’mustcoverthe

abstraction‘‘c’’of‘‘requireddata’’!«Similarityfunction:ca 2 Current:Incollaboration,theabstraction‘‘b’’ofthe

‘‘delivereddata’’bycommunicationmustcoverthe

(10)

Validationoftheprocessmodel:Theinstantiatedprocessis validatedusingEMF’smodelvalidation tool,whichchecks both ECOREstructuralconstraintsandadditionalOCLconstraints.

Identiﬁcation of the pairwise collaborations (PwCs): The process model (instantiated, validated and completed) is then studiedinordertoidentifythePwCs(seeSection3.3).Tothisend,a zoomisnecessaryonthe‘‘CollaborativeWork’’classintheprocess model.AsitisillustratedinFig.10,thisclasshasonlyoneinstance: CW06-07intheprocessmodelofthecase.Thisshowsthatone PwCexistsintheprocess.

Extractionofexchangeddataformats:ForCollaborativeWork CW06-07,studyingtheprocessmodel,theexchangeddataﬁles (produced,deliveredandrequired)shouldbeidentiﬁedinorderto extracttheirformats:

DF06_Pwhichistheproduceddataﬁle.Theformatofthisﬁleis basedon‘‘MSVisioDrawing’’and‘‘SK2XML’’respectivelyinthe initialcaseandthealternative.

DF06_Dwhichisthedelivereddataﬁle.Theformatofthisﬁleis basedon‘‘MSVisioDrawing’’and‘‘STEPXML’’respectivelyinthe initialcaseandthealternative.

Fig.9.Partialdesignprocessincasestudyincludinginitialandalternativeprocesses.

(11)

DF06_Rwhichistherequireddataﬁle.Theformatofthisﬁleis basedon‘‘STEPXML’’intheinitialcaseandthealternative.

Syntaxmatchingofdataformats:Theidentiﬁedformats,in theuniquePwCofthecase,areusedtodeﬁnethesyntaxaspectof DataAdequacy(seeSection3.4)incollaborationCW06-07.Thisis performedviatwosyntaxmatching:

Syntaxmatchingatpotentiallevelindicatingthecompatibility between the formats of data ﬁles DF06_P (produced data) and DF06_R(requireddata)

Currentlevelindicatingthecompatibilitybetweentheformats ofdataﬁlesDF06_P(produceddata)andDF06_R(requireddata). Giventhesizeofthiscasestudy,bothautomaticandmanual syntaxmatching approaches (see Section3.4.A) canbe applied whichprovidedthesameresults.Theseresultsaresummarizedin

Table 3.

Extraction of exchanged data abstractions: After syntax matching,theexchangedﬁles,identiﬁedpreviouslyfor Collabora-tiveWorkCW06-07,areonceagaininvestigatedinordertoverify theavailability of their abstractions. In the case, thefollowing abstractionswereavailable:

ADF06_P,whichistheabstractionofproduceddata,ﬁleDF06_P. Thisabstractionisbasedonthemeta-modelof‘‘MSVisioDrawing’’ andmeta-modelof‘‘SK2XML’’(XSD)respectivelyintheinitialand alternativecases.

ADF06_D, which is the abstraction of delivered data, ﬁle DF06_D.Thisabstractionisbasedonthemeta-modelof‘‘MSVisio Drawing’’andmeta-modelof‘‘STEPXML’’respectivelyintheinitial andalternativecases.

ADF06_R,whichistheabstractionofrequireddata,ﬁleDF06_R. Thisabstractionisbasedonthemeta-modelof‘‘MSVisioDrawing’’ and meta-model of ‘‘STEP XML’’ respectively in theinitial and alternativecases.

Semanticmatchingofdataabstractions:Theaboveidentiﬁed abstractions(availablealsoasECOREmeta-models)wereusedin this step of methodology in order to increase the precision of interoperabilitymeasurement:

Semanticmatchingatpotentiallevelindicatingthe compati-bilitybetweentheabstractionsofdataﬁlesADF06_PandADF06_R, whichrespectivelyplaytheroleofsource1 andtarget.

Semanticmatchingatcurrentlevelindicatingthecompatibility between the abstractions of data ﬁles ADF06_D and ADF06_R whichrespectivelyplaytheroleofsource2 andtarget.

Given the size of the above abstractions, the automatic matchingprocesswasapplied(seeSection3.4.B).Thisstepstarts withtheselectionoftwoabstractions,alreadyimportedinEclipse asECOREmeta-models,asthesourceandtarget.Thenusingthe EMFComparetool,theresultsofthecomparisonarepresentedas structuraldifferencesbetweentheabstractions.

Theresultsofsemanticmatchingforthealternativecaseare includedinTable 4. Inthistable,thedesignerrequiringthedata deﬁnesthethreshold(T)basedontheminimumdatarequiredfor realizinghisactivity.Inthecasestudy,thisvariableissetto0.5, whichindicatesthatatleasthalfoftherequireddatashouldbe deliveredtothedesignerafterthedataexchange.

4.2. Finalresults

Theresultsofsyntaxandsemanticmatchingforbothinitial andalternativecasesaresummarizedinTable 5. Considering theresultsindicatedinthistable,itisobservedthatintheinitial case, the performance indicator has an unsatisfying value at potentialandcurrentlevels.Inthealternativecase,thevalueof the indicatorat the currentlevel is improved. These observa-tions can be detailed. Regarding the alternative case, a pre-developed modeltransformationis applied basedonsemantic relations between product models. Such transformation is capable of ensuring the semantic aspect of interoperability. Meanwhile, the transformed models must be based on a common language,mainly XMI or XML, to ensure the syntax aspects.

Inthecasestudy,thealternativeaffectsthepotentialvalueof syntax matching. In fact, this alternative affects not only communication, but alsothechoiceof design tools.The model transformation requires changing the MS Visio tool to Eclipse modeleditor.Suchchangeinadesignprocessmightbeconsidered asaradicalchange.Therefore,thedecision-makingisimportantin thiscase,whichrequiresfurtherstudiesontheothercriteriasuch ascostortimeofimplementationofmodeltransformation. Table3

Resultsofsyntaxmatchingforinitialandalternativecases.

Input Rule1(potentiallevel) Rule2(currentlevel)

OCL Manuel OCL Manuel

CollaborationCW06-07 (initialcase)

Unsatisfying Unsatisfying Unsatisfying Unsatisfying (MSVisiodrawing6¼STEPXML) (MSVisiodrawing6¼STEPXML)

CollaborationCW06-07 (alternative)

Unsatisfying Unsatisfying Satisfying Satisfying

(SK2XML6¼STEPXML) (STEPXML=STEPXML)

Table4

Sampleresultsofsemanticmatchingforalternativecase.

Class Variable Classproperty Variable Proprietyofpropriety Variable Attribute Association

ElementsintargetADEF06_R 13 NC 47 16 NP (47+16)3=189 NPP

Mismatchesaccordingtosource1ADEF06_P 13 NMC1 47 16 NP1 (47+16)3=189 NMPP1 Mismatchesaccordingtosource2ADEF06_D 5 NMC2 14 5 NP2 (14+5)3=57 NMPP2

Weights 1 WC 0.5 0.5 WP 0.1 WPP

Threshold(T)=0.5

Weightoftargetselements(WE) 131+(47+16)0.5+1890.1=63.4 Weightofmismatchesinsource1(WME1) 131+(47+16)0.5+1890.1=63.4 Weightofmismatchesinsource2(WME2) 51+(14+5)0.5+570.1=20.2 Semanticmatchingvalue(potentiallevel) 163:4

63:4¼0<T!Unsatisfying

Semanticmatchingvalue(reallevel) 120:2

(12)

5. Conclusions

In this paper, it is considered that collaboration between designersisrealizedthrough theexchange ofdatafiles.In such collaborations,alowlevelofperformancecancauseproblemsof data adequacy that can potentially cause failures in designed products.Regardingthe evaluationof the collaborations’ perfor-mance,anapproachisstillmissingthatstudiessuchperformancein termsofbothconceptualandtechnicalbarriersofinteroperability since analyzing the collaborations as white boxes. Therefore, a tooledmethodologyisproposedinthispapertofillthegapidentified basedontheliteraturereview.Theinnovationsandadvantagesof thismethodology,illustratedbythiscasestudy,areprimarily:

Theadaptabilityoftheproposedmethodologytocollaborative environmentsmightbeitsmainadvantagesinceitselementsare basedonthestateoftheartoftheconceptsofcollaborationand interoperability.Infact,themethodologyisbasedontheseconcepts todistinguishthecollaborationperformancefromtheperformance ofindividualactivitiesandfromtheperformanceevaluationsbased onclassicalcriteriasuchascostortime.Thisreducesthe riskof interferenceofparametersthatareoutsidethecollaborationscope. Forexample,inthemodelingphase(phase1a.1),amongallprocess elements,onlytheelementsthataffect thecollaboration perfor-mancearepreserved.Moreover, inthe measuring phase (phase 1a.2),theindicatorisderivedfromtheconceptofinteroperability. Theautomationisincreasedinseveralwaysinthismethodology. For example, in the modeling phase, using the meta-model of collaboration developed in Eclipse, it is possible to generate automaticallyaninitialinstanceoftheprocessmodel.Thevalidation ofthesyntaxofthismodelcanalsobedoneautomatically.Moreover, inthemeasuringphase,usingOCLandEMFComparetool,respectively thedeﬁnedsyntaxandsemantic matchingrulescan beveriﬁed. Onone hand,theseexamplesofautomationcanreducetheriskofhuman errororjudgmentsinapplyingtheapplicationofmethodologyandon theotherhand,theycanreducetheexecutiontime.

Integratingtechnicalandconceptualaspectsof interoperabili-ty:Collaborationperformanceisevaluatedbasedon interopera-bility at two levels: technical and conceptual. Considering the complementaryrole of these levels to achieve interoperability in the exchanged data, in the methodology both aspects are integratedinthemeasurementphase.

Interactionwithstakeholders:severalinteractionsbetweenthe user of the methodology and stakeholders (designers, process management)areconsideredinthemeasurementphasetohavea bettertranslationoftheresultsbasedonobjectiveoftheprocess.For example,thethresholdofsatisfactionindataadequacyisbasedon theopinionofthedesigners.Indeed,thedesignerdeﬁnestowhat extentthedelivereddataareadequateforperformingitsactivities. Regarding the perspectives of this paper, it is primarily suggestedtostudytheapplicabilityoftheproposedmethodology

foritspotentialusers.Thismethodologyrequirestheavailabilityof technical arrangements such as Eclipse installations and basic knowledgeofdatamodeling.Inaddition,theintelligibilityoftasks andresultsforusersmustbestudied.Thisallowsdeﬁningthelevel ofuserautonomyinperformingthevarioustasks.Aslongasthe keyfeatureofthismethodology,itsapproach,isrespected,itis possible to evolve or upgrade the current choice of tools or methods.Suchmodiﬁcationscanbedonebasedonthecaseunder evaluation. This might also increase the accuracy of results. Currentlyinthemodelingphase(Phase1a.1),thegranularityis limited to pairwise collaborations. On one hand, collaboration between multipleactorswithmultipledataexchangesmustbe studied.On theotherhand,theimpactofperformance ofeach dataexchange on theoverallvalue oftheprocess performance mustbeinvestigated.Regardingthemeasuringphase(Phase1a.2), the similarity functionsin matching can be replaced by more advanced functions [67,85]. For semantic matching via EMF Comparetool,anevolutionmaybealsoconsideredbyintegration ofexpertopinionordomainontologyinthematchingprocess[86].

Acknowledgment

Thisresearchpaperisaresultofthecorrespondingauthor’sPhD thesis,performedatLCFCandLSISlaboratoriesofEcoleNationale Superieured’ArtsetMetiers(ENSAM-ArtsetMetiersParisTech) France.ThisworkhasbeenfundedbyENSAM.

References

[1]J.S.Gero,Designprototypes:aknowledgerepresentationschemafordesign,AI Magazine11(4)(1990)26–36.

[2]G. Pahl,W. Beitz, in:Ken Wallace (Ed.),EngineeringDesign: A Systematic Approach,London,UK,Springer-Verlag,1996.

[3]F.Mistree,W.F.Smith,B.Bras,J.K.Allen,andD.Muster,‘‘Decision-BasedDesign: AContemporaryParadigmforShipDesign,’’inAnnualMeeting,1990. [4]I.M.M.J.Reymen,ImprovingDesignProcessesthroughStructuredReﬂectiona

DomainDependentApproach,EindhovenUniversityofTechnology,Eindhoven, Netherlands,2001(PhDThesis).

[5]T.J.Howard,S.J.Culley,E.Dekoninck,Describingthecreativedesignprocessbythe integrationofengineeringdesignandcognitivepsychologyliterature,Design Studies29(2(Mar))(2008)160–180.

[6]A.Larsson,EngineeringKnow-Who:Whysocialconnectednessmatterstoglobal designteams,LuleaUniversityofTechnology,Lulea,Sweden,2005(PhDThesis).

[7]F.Danesi,N.Gardan,andY.Gardan,‘‘Collaborativedesign:fromconceptto appli-cation,’’inProceedingsoftheGeometricModelingandImagingGMAI’06,2006. [8]L.Wang,W.Shen,H.Xie,J.Neelamkavil,A.Pardasani,Collaborativeconceptual

design—stateoftheartandfuturetrends,ComputerAidedDesign34(2002)981–996.

[9]D.J.Wood,B.Gray,Towardacomprehensivetheoryofcollaboration,TheJournal ofAppliedBehavioralScience27(2)(1991)143–149.

[10]W.L.Bedwell,J.L.Wildman,D.DiazGranados,M.Salazar,W.S.Kramer,E.Salas, Collaboration at work: an integrative multilevel conceptualization, Human ResourceManagementReview22(2(Jun))(2012)128–145.

[11]M.L.Chiu,Anorganizationalviewofdesigncommunicationindesign collabora-tion,DesignStudies23(2)(2002)187–210.

[12]R.Valkenburg,K.Dors,Thereﬂectivepracticeinproductdesignteams,Design Studies19(2000)249–271.

[13]L. Yan,Z.Li, X.Yuan, Studyon method-of-robust-multidisciplinary-design-collaborative-decision for productdesign,InformationTechnologyJournal8 (4)(2009)441–452.

[14]T.Paviot,V.Cheutet,S.Lamouri,APLCSframeworkforPDM/ERPinteroperabilty, InternationalJournalofProductLifecycleManagement5(2/3/4)(2011)295–313.

[15]F. Giunchiglia, P.Shvaiko, Semanticmatching, The Knowledge Engineering ReviewJournal18(3)(2004)265–280.

[16]R.Spalazzese,‘‘TowardsMediatorConnectorsforApplicationlevel Interoperabili-tyCategoriesandSubjectDescriptors,’’inESEC/FSEDoctoralSymposium’09, 2009,vol.09,pp.35–36.

[17]V.Fortineau,T.Paviot,S.Lamouri,Improvingtheinteroperabilityofindustrial informationsystemswithdescriptionlogic-basedmodels—thestateoftheart, ComputersinIndustry64(4(May))(2013)363–375.

[18]G.WiederholdandM.Genesereth,‘‘TheConceptualBasisforMediationServices GioWiederholdandMichaelGeneserethAttachments:TheConceptualBasisfor MediationServices,’’no.October1996,pp.1–23,1997.

[19]C.H.Goh,S.E.Madnick,M.D.Siegel,S.Wp,C.Hian,G.Stuart,Ontologies,Contexts, andMediation:RepresentingandReasoningaboutSemanticsConﬂictsin Het-erogeneousandAutonomousSystems,TheSloanSchoolofManagement Mas-sachusettsInstituteofTechnologyCambridge,Cambridge,Massachusetts,United States,1995(Workingpaper3848CISL-Technicalreport95-04,’’no.August).

Table5

Summaryofcasestudyresults. Phase1a.1:Modeling

Onepare-wisecollaborationintheprocessisfound,knownbytheclass CollaborativeWorkCW06-07

Phase1a.2:Measurement(forcollaborationCW06-07)

Initialcase Alternativecase Syntaxmatching Potential Unsatisfying*

Satisfying* Current Unsatisfying*

Satisfying* Semanticmatching Potential Unsatisfying Unsatisfying**

Current Unsatisfying Satisfying** Performanceof

collaboration

Potential Unsatisfying Unsatisfying Current Unsatisfying Satisfying

*

ResultsareprovidedfromTable3.

**

(13)

[20]W.I.M.Y.W.Hsu,Currentresearchintheconceptualdesignofmechanical pro-ducts,ComputerDesign30(5)(1998)377–389.

[21]V.R.M.Kleinsmann,J.Buijs,Understandingthecomplexityofknowledge integra-tionincollaborativenewproductdevelopmentteams—acasestudy,TheJournal ofEngineeringandTechnologyManagement27(2010)20–32.

[22]M.Kajtazi,D.Haftor,A.Mirijamdotter,InformationInadequacy:SomeCausesof FailuresinHuman,SocialandIndustrialAffairs,TheElectronicJournal Informa-tionSystemsEvaluation14(1)(2011)63–72.

[23]F.-A.GruatLaForme,V.B.Genoulaz,J.-P.Campagne,Aframeworktoanalyse collaborativeperformance,ComputerinIndia58(7(Sep))(2007)687–697.

[24]C.Bock,X.Zha,H.Suh,J.-H.Lee,Ontologicalproductmodelingforcollaborative design,AdvancedEngineeringInformatics4(2010)510–524.

[25]G.Backlund,A.O.Ronnback,Managingcomplexityincollaborativedevelopment modelingrequirements andenhancingcommunication,Microprocessorsand Microsystems23(7(Dec))(1999)409–416.

[26]C.Romito,D. Probert,C.Farrukh,Technologyevaluationunder information inadequacy,in:PICMETProceedings,2007,pp.1672–1679.

[27]L.A.Shah,Value-RiskbasedPerformanceEvaluationofIndustrialSystems,E´cole NationaleSupe´rieured’ArtsetMe´tiers,Metz,France,2012(PhDThesis).

[28]M.Bosch-Mauchand,A.Siadat,N.Perry,A.Bernard,VCS:valuechainssimulator,a toolforvalueanalysisofmanufacturingenterpriseprocesses(avaluebaseddecision supporttool),JournalofIntelligentManufacturing23(4)(2012)1389–1402.

[29]K.Husted,S.Michailova,DualallegianceandknowledgesharinginInter-ﬁrm R&Dcollaborations,OrganizationalDynamics39(1(Jan))(2010)37–47.

[30]R.Ha¨ma¨la¨inen,K.Va¨ha¨santanen,Theoreticalandpedagogicalperspectiveson orchestratingcreativityandcollaborativelearning,EducationalResearchReview 6(3(Jan))(2011)169–184.

[31]W.A.Hatem,A.Kwan,J.Miles,Comparingtheeffectivenessoffacetofaceand computermediatedcollaboration,AdvancedEngineeringInformatics26(2(Apr)) (2012)383–395.

[32]M.Schrage,SharedMinds:TheNewTechnologiesofCollaboration, Random House,NewYork,1990.

[33]W.Li,Y.Lin,Y.Liu,Thestructureofweightedsmall-worldnetworks,PhysicaA: StatisticalMechanicsanditsApplications376(2007)708–718(Mar).

[34]P.Girard,V.Robin,Analysisofcollaborationforprojectdesignmanagement, ComputersinIndustry57(8-9(Dec))(2006)817–826.

[35]N.Daclin,D.Chen,andB.Vallespir,‘‘Enterpriseinteroperability measurement-Basicconcepts,’’inProceedingsofthe3rdWorkshoponEnterpriseModelingand OntologiesforInteroperability,2006.

[36]E.Yahia,A.Aubry,H.Panetto,Formalmeasuresforsemanticinteroperability assessmentincooperativeenterpriseinformationsystems,Computersin Indus-try63(5)(2012)443–457.

[37]IEEE,‘‘StandardComputerDictionary-ACompilationofIEEEStandardComputer Glossaries,’’1990.

[38]EU,‘‘TheRoleofeGovernmentforEurope’sFuture-Report,’’COM,vol.567,2003. [39]D.ChenandN.Daclin,‘‘Frameworkforenterpriseinteroperability,’’inIFAC2nd InternationalWorkshoponEnterpriseIntegration,Interoperabilityand Network-ing(EI2N’2006),2006.

[40]G.Creese,Informationscarcitytoinformationoverload,enterprisecontent man-agement,DMReview(2007(Jan))20–22.

[41]S.Bassetto,A.Siadat,Operationalmethodsforimprovingmanufacturingcontrol plans:casestudyinasemiconductorindustry,JournalofIntelligent Manufactur-ing20(1)(2009)55–65.

[42]A.Pirayesh,A.Siadat,R.Tavakkoli-Moghaddam,andF.Jolai,‘‘Anintegrated approachforrisk-assessmentanalysisinamanufacturingprocessusingFMEA andDES,’’in2011IEEEInternationalConferenceonQualityandReliability,2011, pp.366–370.

[43]ScopusElsevier,‘‘Key:interoperability:1981to2012,AnalyzeResultsbysubject areaonlinetool.’’hhttp://www.scopus.com/i,2013(accessed15.03.13). [44]M.Kasunic,W.Anderson,Interoperability:ChallengesandOpportunities,

Car-negie-MellonUniversity—SoftwareEngineeringInstitute,Pittsburgh,PA,2004.

[45]S.Blanc,Y.Ducq,B.Vallespir,Evolutionmanagementtowardsinteroperable supplychainsusingperformancemeasurement,ComputerinIndustry—Elsevier 58(7)(2007)720–732.

[46]M.Camara,Y.Ducq,andR.Dupas.,‘‘MethodologyforPriorEvaluationof Inter-operability,’’inCollaborativeNetworksforaSustainableWorld,Proceedingsof the11thIFIPWG5.5WorkingConferenceonVirtualEnterprises,PRO-VE2010., 2010,pp.697–704.

[47]D.Chen,B.Vallespir,andN.Daclin.,‘‘Anapproachforenterpriseinteroperability measurement,’’ inProceedings oftheworkshopMoDISE-EUS, ModelDriven Information Systems Engineering: Enterprise, User and System Models, MoDISE-EUS’08,2008.

[48]D.Bianchini,V.DeAntonellis,M.Melchiori,andD.Salvi,‘‘Semantic-enriched servicediscovery,’’inICDEW’06Proceedingsofthe22ndInternational Confer-enceonData,2006,pp.38–48.

[49]M.Barut,W.Faisst,J.J.Kanet,Measuringsupplychaincoupling:aninformation systemperspective,EuropeanJournalofPurchasing&SupplyManagement8(3) (2002)161–171.

[50]Y.DucqandD.Chen.,‘‘Howtomeasureinteroperability:Conceptand Ap-proach,’’inProceedingsofthe14thInternationalConferenceonConcurrent Enterprising:ICE2008ANewWaveofInnovationinCollaborationNetworks, 2008,pp.583–590.

[51]T.Ford,J.Colombi,S.Graham,andD.Jacques,‘‘TheInteroperabilityScore,’’in Proceedingofthe5thConferenceonSystemsEngineeringResearch,2007. [52]G.LaVean,Interoperabilityindefensecommunications,IEEETransactionson

Communications28(9)(1980)1445–1455.

[53]D.R.Mensh,R.S.Kite,P.H.Darby,TheQuantiﬁcationofInteroperability,Naval EngineersJournal(WileyOnlineLibrary)101(3)(1989)251–259.

[54]M.Amanowicz,andP.Gajewski.,‘‘Militarycommunicationsandinformation systemsinteroperability,’’inMilitaryCommunicationsConference,MILCOM’96, ConferenceProceedings,IEEE,1996,vol.1,pp.280–283.

[55]C4ISR,‘‘ArchitectureWorkingGroupFinalReport,’’no.April.Departementof Defence,USA,1998.

[56]M.J. Leite,‘‘InteroperabilityAssessment,’’inProceedingsofthe 66thMORS SymposiumNaval,PostGraduateSchool,1998

[57]T.ClarkandR.Jones,‘‘OrganisationalinteroperabilitymaturitymodelforC2,’’In ProceedingsoftheCommandandControlResearchandTechnologySymposium (CCRTS),1999.

[58]A.Tolk,‘‘Beyondtechnical interoperability-Introducingareferencemodelfor measuresofmeritforcoalitioninteroperability,’’inProceedingsofthe8th Interna-tionalCommandandControlResearchandTechnologySymposium(ICCRTS),2003. [59]ATHENA,‘‘AdvancedTechnologiesforInteroperabilityofHeterogeneousEnterprise

NetworksandtheirApplications,’’inFP6-2002-IST-1,IntegratedProject,2003. [60]ISOFDIS11354-1,‘‘TC184/SC5:Advancedautomationtechnologiesandtheir

applications-Requirementsfor establishingmanufacturing enterpriseprocess interoperability-part1:frameworkforenterpriseinteroperability.’’ISO,Geneva, Switzerland,2010.

[61]W.Gue´dria,Y.Naudet,D.Chen,InteroperabilityMaturityModels–Surveyand Comparison,in:OTMConfederatedInternationalWorkshopsandPostersonOn theMovetoMeaningfulInternetSystems,Springer,2008,pp.273–282.

[62]V.Issarny,A.Bennaceur,andY.-D. Bromberg,‘‘Middleware-layerConnector Synthesis:BeyondStateoftheArtinMiddlewareInteroperability,’’in11th InternationalSchoolonFormalMethodsfortheDesignofComputer, Communi-cation and Software Systems:Connectors for Eternal Networked Software Systems,2011,pp.217–255.

[63]M.Ehrig,Y.Sure,Ontologymapping—anintegratedapproach,LectureNotesin ComputerScience3053(2004)76–91.

[64]J.Euzenat,T.LeBach,J.Barrasa,P.Bouquet,J.DeBo,R.Dieng,M.Ehrig,M. Hauswirth,M.Jarrar,R.Lara,D.Maynard,A.Napoli,G.Stamou,Heiner, Stuck-enschmidt,P.Shvaiko,SergioTessaris,S.VanAcker,andI.Zaihrayeu,‘‘D2.2.3: Stateoftheartonontologyalignment,KWEBEU-IST-2004-507482Project,v1.2.’’ KnowledgeWebConsortium,2004.

[65]Y.Lu,ApproachforInformationSystemsSemanticInteroperabilityinSupply ChainEnvironment,ZhejiangUniversity,Hangzhou,ZhejiangProvince,China, 2012(PhDThesis).

[66]J.Euzenat,P.Shvaiko,Ontology Matching, 2nded., Springer,2013,http:// dx.doi.org/10.1007/978-3-642-38721-0.

[67]E.Rahm,‘‘Towards large-scale schema and ontologymatching,’’ inSchema MatchingandMapping,Data-CentricSystemsandApplications,2011,pp.3–27. [68]D.S.Kolovos,Establishingcorrespondencesbetweenmodelswiththeepsilon comparisonlanguage,LectureNotesinComputerScience5562(2009)146–157.

[69]M.StephanandJ.R.Cordy,‘‘ApplicationofModelComparisonTechniquesto ModelTransformationTesting,’’inMODELSWARD2013-InternationalConferenc onModel-DrivenEngineeringandSoftwareDevelopment,2013,pp.307–311. [70]Eclipse,‘‘Epsilon,’’2013. [Online].Availablefrom:hhttp://www.eclipse.org/i.

[Accessed:10.08.13].

[71]A.Toulme´,‘‘PresentationofEMFCompareUtility,’’inEclipseModeling Sympo-sium,2006,pp.1–8.

[72]ISO,‘‘ISO10303-1:1994IndustrialautomationsystemsandintegrationProduct datarepresentationandexchangeoverviewandfundamentalprinciples, Inter-nationalStandard,ISOTC184/SC4,’’1994.

[73]S. Rachuri, E.Subrahmanian, A.Bouras, S.J.Fenves, S. Foufou,R.D. Sriram, Informationsharingandexchangeinthecontextofproductlifecycle manage-ment:roleofstandards,ComputerDesign40(7(Jul))(2008)789–800.

[74]S.Terzi,ElementsofProductLifecycleManagement:Deﬁnitions,OpenIssues, l’Universite´ HenriPoincare´,Nancy-I,Nancy,France,2005(PhDThesis).

[75]STEPToolsInc.,‘‘WhatisSTEP?,’’Copyrightß1991-2013STEPToolsInc.,.hhttp:// www.steptools.com/i,2013.

[76]A.Tursi,Ontology-basedApproachforProduct-DrivenInteroperabilityof Enter-priseProductionSystems,Universite´ HenriPoincare´,PolitecnicodiBari,2009 (PhDThesis).

[77]S.AbdulGhafour,P.Ghodous,andB.Shariat,‘‘IntegrationofProductModelsby OntologyDevelopment,’’in13thInternationalConferenceonInformationReuse andIntegration(IRI),2012IEEE,2012.

[78]Y.Liao,M.Lezoche,H.Panetto,andN.Boudjlida,‘‘SemanticAnnotationModel DeﬁnitionforSystemsInteroperabilityWhatisSemanticAnnotation?,’’inOTM 2011 Workshops20116thInternationalWorkshoponEnterpriseIntegration, InteroperabilityandNetworking(EI2N),2011.

[79]J.MillerandJ.Mukerji,‘‘MDAGuideVersion1.0.1,’’no.June,2003.

[80]K.CzarneckiandS.Helsen,‘‘ClassiﬁcationofModelTransformationApproaches,’’ inOOPSLA’03WorkshoponGenerativeTechniquesintheContextof Model-DrivenArchitecture,2003.

[81]I.Kurtev,K.vandenBerg,MISTRAL:Alanguageformodeltransformationsinthe MOFmeta-modeling architecture,LectureNotes inComputerScience3599 (2005)139–158.

[82]Eclipse,‘‘Eclipse:Ecore.’’hhttps://wiki.eclipse.org/Ecore_Toolsi,2014. [83]OMG,‘‘OMG’sMetaObjectFacility.’’hhttp://www.omg.org/mof/i,2012(accessed

31.03.13).

[84]M.Iraqi-houssaini,M.Kleiner,L.Roucoules,Model-based(Mechanical)product design,LectureNotesinComputerScience6981(2011)548–562.

[85]P.Shvaiko,Asurveyofschema-basedmatchingapproaches,JournalonData SemanticsIV(2005)146–171.

(14)

[86]M.LimaDutra,Anontology-basedapproachtomanageconﬂictsincollaborative design,UniversidadenovadeLisboa,Lyon,2009(The`sedel’Universite´ Claude Bernard).

AmirPirayeshNeghabiscurrentlyaresearchengineer intheInternationalVirtualLaboratoryforEnterprise Interoperability (InterOP-VLab), Talence, France. He obtainedhisPhDinindustrialandmechanical engi-neering from Ecole Nationale Supe´rieure d’Arts et Me´tiers,Francein2014. Hisresearchinterestsinclude Interoperability Evaluation, Performance Measure-ment,RiskManagement,DataModelingand Simula-tion,inproductlifecycle.

AlainEtienneisanassociateprofessorofcomputerand industrial engineering in the Laboratory of Design, ManufacturingandControl(LCFC)atEcoleNationale Supe´rieured’ArtsetMe´tiers,France.Heobtainedhis PhDinindustrialengineeringfromUniversityofNancy, Francein2007.Hisresearchinterestsincludeartiﬁcial intelligence,informationsystem,riskmanagementand knowledgeformalizationappliedtodecision-making, designofbothproductsandmanufacturingsystems.

MathiasKleinerisanassociateprofessorofcomputer scienceintheLaboratoryofScience,Informationand Systems(LSIS)atEcoleNationaleSupe´rieured’Artset Me´tiers,France.HeobtainedhisPhDfromUniversityof Me´diterrane´e,Marseille,Francein2007.Hisresearch interestsinclude artiﬁcial intelligence, graphs, con-straint programming and metamodeling, and the applicationofthesetechnicstomechatronicproducts design.

Lionel Roucoules is currently professor at Arts et Me´tiersParisTechInstitute(France–www.ensam.eu) sinceSeptember2008.Hedevelopshisresearchinthe InformationandSystemScienceLaboratory(www.lsis. org).Thecontextofhisresearchisintegrateddesign andcollaborativeITplatforminaglobalPLMvision.His speciﬁc interest is product-process interface. He proposeda DFM-synthesis approach,which is now partof larger DFX by least commitmentmodeling conceptssupported bya Model DrivenEngineering platformforVirtualProductDevelopment.