Experiences in Applying Artificial Intelligence within SIARAS Project

(1)

within SIARAS Project

SªawomirNowaczyk

KatedraAutomatyki,

AkademiaGórniczo-Hutnicza,

[email protected]

Abstract. In this paper we present our experience in designing and

buildingknowledgebasefortheSIARASproject.Thisprojectaimedto

produceatool,calledSkillServer,supportingengineersduringrecong-

urationofmanufacturingsystems.Wemainlyfocusontheprocessofde-

velopingknowledgerepresentationusedbySkillServeranddiscusssome

lessonswe have learnedabout applying Articial Intelligence concepts

inpractical applications within high-techindustry.We discuss several

typesof knowledgethat Skill Serverneeds to have access to,and how

dierentKRsolutionscanbeintegratedtogetherinacoherentsystem.

1 Introduction

In this paper we present our experience in designing and building knowledge

basefortheSIARASproject,focusingespeciallyonhowtheapproachesweused

evolvedaswegainedmoreandmoreinsightintotheactualrequirementsofthe

system.

SIARAS is an acronym of an EU-funded (FP6 - 017146) STREP-project

Skill-BasedInspectionandAssemblyforRecongurableAutomationSystems,

running in theyears 20062008.Its generalaim wasto support end users and

engineers of manufacturing systems, including robotic ones, and to makepro-

ductionengineeringeasier(andthuscheaper)inseveralcommoncircumstances.

The primary goal of the project was to build an intelligent system, pro-

visionally called Skill Server, that would be capable of supporting automatic

and semi-automatic reconguration of manufacturing processesin response to

changing requirements.The main issue during the design phase wasto merge

two, somewhat opposed, views on the reconguration process: the top-down,

AI-basedapproachandthebottom-up,engineeringone.

Thetop-downapproachdescribestherecongurationasa(re)planningprob-

lem. Weare given anew task, usually expressed asa goal condition,possibly

beingamodicationofanearlier,correctone.Given aset ofskills availablein

the system, understood as adescription of the operations that might be per-

formed by the devices available to the user, we are to nd such asequence of

operationsthatwillensurethatthetaskiscorrectlyexecuted,i.e.tondaplan

(2)

of an existing, correct, properly modelled production line. The system is not

expected to propose novel solutions,nor to search for alternative ways of im-

plementing theprocess. Inparticular, oneshould expectadetailed description

of thetask:what is producedand how(i.e. what arethestepsof theprocess).

Moreover,foreachstep,itshouldbeclearhowdoesitcontributetothegoal.On

theother side, availabledevices must be describedin termsof operationsthey

are abletoperform(skills)and conditionsunder which theycanoperate.Skill

Serverneedstomaptaskintoskills andparametrisethemappropriately.

It is rather obvious that the top-down AI approach is both computation-

ally infeasible and impossible to model suciently well, while the bottom-up

reparametrisationapproachlacks generalityand risksending upasadatabase

of previously used parameter settings for a number of devices in a number of

scenarios.Themainissuewiththisapproachisguaranteeingscalabilityandex-

tendibilitytonewdomainsortonewkindsofdevices.Thereisariskoflimiting

theapproachtothepreviouslyconsideredcasesandverysimilaronesonly,thus

precludingamoreopen-endedsolution.

Takingthisintoaccount,wehavesettledforalayeredapproach,withrecon-

gurationlevelat thebottomand(re)planninglevelontopofit.

2 Knowledge Representation

Initially,wehaveidentiedseveraltypesofknowledgeSkillServerwilluse:skills,

devices,tasks,workpiecesandenvironment.Mostofthemcanbespeciedon,at

least, twolevelsofabstraction:simplied,generic descriptions(likeauniversal

pickup skill) and instantiated ones (the operation of gripper

G ₁

^picking ^the

windshield

W ₁

ⁱⁿ ^factory

F

^, ^at ^time ^point

t n

^and ^position

p m

^). ^Throughout

the project, however, we have been continuously investigating possibilities of

introducingadditional,intermediatelevelsofabstraction inbetween.

Nevertheless, in addition to the symbolicknowledge, there exist anumber

of domain-specic ordevice-specic procedures for calculating various aspects

(e.g. trajectory planner, device calibration and reparametrisation procedures,

etc.) which, in many contexts, should also be treated as knowledge. Despite

severalattempts,however,wehavenotmanagedtondanacceptableanduseful

wayto put them into anykind ofevensemi-formal framework.This would be

averyinterestingresearchprojectin itself, butour experienceshowsthat this

knowledgeissimplytoodiversetoformaliseinthetimeframewehad.Therefore,

SkillServerconsiderssuchprocedures as,essentially,blackboxes.

Theoverallstructure of theSkillServerisshownin Figures 1and 2.They

depict,fromtwodierentpointsofview,howthecoreSkillServerisintegrated

withvariousknowledge,reasoninganduserinterfacemodulesandwhatkindof

informationsourcesithasat itsdisposal.

Ineect,wehavedevotedmostofoureortstodesigningsymbolicknowledge

representation in a way that would be intuitive and useful in practice to our

(3)

control pgm

distributed library HTTP access?

CAD sim pgm skills devices instantiated skills

actual task

THE SKILL SERVER

representation

user interface for task definition

skill server main loop

plug−ins registration?

interface?

operations virt tasks virt workpiece ???

visualisation

simulation

user interface for device descriptions

ontology

UF #1 UF#2 UF#n

device library

device#1234 ontology

Fig.1.SkillServerstructure

sametime,allowSkillServertoperformthereasoningtasksrequired.Itturned

outtobeasurprisinglydiculttasktoagree uponaknowledgerepresentation

formalismwithintheconsortium,however.Infact,whilewe(fromtheacademia

side)werealreadyawarethat therearecostsassociatedwithformalapproaches

to KR, we have not anticipated how dicult it will be to convinceindustrial

partnersthat thebenetsoutweighthosecosts.

Inprinciple,theinitialresponsewasthatanyrepresentationmoreadvanced

than attribute-value pairs was deemed as unnecessarily complex and still not

expressiveenough.Basically,theexpectationsseemedtobethatweshoulduse

attribute-valuepairswhereverappropriate,andfull expressiveness ofprogram-

ming languageseverywhere else.Finally, theconsortiumhaveeventually man-

aged to agree upon using Ontology asadefault knowledge representation for-

(4)

Time optimisation

Grippability analysis

Vendor specific Ontology Database

Commercial

Custom reasoner

Main integrator

Open source software

Commercial software

optimisation loop

Energy reasoner

System

planning Path

provider

Device End user

reasoner SIARAS

Utility function interfaces (G)UI interfaces

Reasoner interfaces Simulation/visualisation interfaces

Fig.2.SkillServerstructure

Oneoftheissuesthat causedthemostdiscussions amongtheprojectpart-

nerswasthecontentsandstructureoftheontology.Thelatterquestionismostly

technicalandamountstoxingarepresentationwhichwouldmaketypicaluses

oftheknowledgestoredin theontologyaseasyaspossible.Still,during discus-

sions,thosetwoaspectsseemedtobeconfusedveryoften.

However,theformerissue,i.e.what shouldthecontentsoftheontologybe,

was of paramount importance. There was little doubt that it should contain

knowledge about skills and about devices providing them. What was not so

obviouswasthestatusoftasks:whethertheyshouldbepresentin theontology

as aseparateentities.

Therearestrongargumentsbothforandagainstsuchsolution.Astasksare

oneofthemajorconceptspresentinthebasicideaoftheSkillServer,theyneed

(5)

thustheSkillServerwould notbeabletoperformsomeofitsbasicfunctions.

However,on the most basic level, tasks necessarily haveto correspond di-

rectly, on the one-to-one basis, with the skills. Otherwise there would be no

possibility for the skill serverto performany kindof matching betweenthem.

Therefore,itisprobablyunwisetoexactlyduplicatethehierarchyofskillswith

acorrespondinghierarchyoftasks.Moreover,thelifetime of tasksisalsosig-

nicantlydierentthan therestof informationwithin theOntology:skills and

devicesarealmosteternal,inasensethattheycanbeinputintotheknowledge

base once and are useful for everybody any time in the future. Tasks, on the

other hand, are dened within a veryspecic context and usually only useful

within aparticularfactoryshopandforalimitedtime.

Theontologycontainsknowledgeabout(abstract)skillsand aboutdevices.

This is an area for which ontology is well-suited, so it is both easy and e-

cienttospecifythat,forexample,aconceptofvacuumgripperisasubconcept

of gripper,which in turn is a subconceptof device. In asimilar manner, a

vacuum-pickupskillisasubconceptofpickupskill.Eventhoughtheexpres-

sivepowerof an ontology goes much further, wedid in fact get most mileage

outof it bytreatingit aslittlemorethana gloriedtaxonomy.This partwas

considered sucientlyeasyand sucientlyusefulbyallpartners within acon-

sortium to makeituniversally accepted.Infact, in someregards theontology

provedtoosuccessful,andtherehasbeensignicantpressuretoputvirtuallyall

knowledgethere, eventhekindsforwhich itisclearlyinappropriate.

Ausefulfeatureof theontologywasthat itallowedus tospecify properties

of each skill and device, with naturalinheritance rules. Thus we were able to

associatemass propertywith theconceptof device and numberofngers

with nger gripper, to specify that mass can be expressed in grams or

pounds,andthatmaximageresolutionmakessenseonlyforcameras,andnot

forrobots.Whatwedidnotmanagetodoistondawaytoexpressthepurpose

ofthosepropertiesinawaythatwouldbeaccessibletothenon-symbolicparts

of knowledge, such as calibration procedure or specialised domain-dependent

algorithms.However,thishasmoretodowithhowunstructuredthisprocedural

knowledgeturned outtobeandlesswiththeontologyitself.

Anotherissuewehavebeendebatingmanytimeswithintheconsortiumbut

nevermanaged toreachasatisfactoryconclusionwastheconceptofcompound

devices.Forexample,whentalkingaboutrobots,itoftenappearsusefultospec-

ify that arobot usesagripper to perform someskill. It only seemsnatural to

expressthatrobot-with-a-grippercanperformmore(orratherdierent)actions

thanrobotalone.However,doingitproperlyturnedoutto beoutsidetherep-

resentationalpowerofanontology,atleastintheformwehaditintheproject.

3 Ontology Structure

We have decided to center knowledge representation around the concepts of

(6)

sessions,asdynamicstructures, specicto aparticular case.Theycanbeseen

as(arguably,quitecomplex)combinationsofskillsandparametersandtherefore

thereisnoneedtohavethemexplicitinthevocabulary.

Thestaticpartoftheknowledgeisrepresentedinanontology:adatastruc-

turestoringallthenecessaryrelationsbetweenthetermsused.Whileontologies

are often used for classicationpurposes, in our casethe classicationis done

when objects(skills ordevices) are introduced in thestructure. The main use

oftheontologyistoallowreasoningaboutskills matchingparticulartasksand

aboutdevicesbeingsuitableforparticularoperations,aswellastostandardise

thenomenclatureusedandtherelationshipsofdierentconcepts.

FortheprototypeofSkillServer,wehavechosentheopensourcetoolProtégé

forontologycreationandmanipulation,togetherwithreasonerssuchasRacer[1],

Fact++[2],Algernon[3]and Pellet[4]. IfSkillServeriseverto enter production

stage,aspecialiseduserinterfacewillneedtobedeveloped,sincegeneralpurpose

toolswesuchasthosearenotappropriateforitsintendedendusers.Theyworked

newithin theSIARASproject,however.

Theontologycontainsthreehierarchies:Devices,SkillsandProperties.The

Device hierarchy species what types of devices do we intend Skill Server to

reasonabout.Wehaveprovideddevicemanufacturerswitharudimentaryinter-

face for introducing theirproducts: theyspecifywhere in the hierarchyshould

aparticular deviceresideand, dependingon that, theyareaskedto llin val-

ues of appropriate properties. Those devices are, at least conceptually, leaves

(instances) of the Device hierarchy. However, since we expect no centralised

repositoryofalldevicesto beavailable andassumethatdata willbeorganised

inadistributedmanner(forexample,downloadablefromdevicemanufacturers'

webpages),weareactuallystoringallthedevicesin anSQLdatabase.

ThesecondhierarchyistheSkillhierarchy,whichmainlyaimsatprovidinga

commonvocabularybetweenthedevicemanufacturers(whospecifywhichskills

doesadeviceoer)andtheendusersorsystemsintegrators(whospecifywhat

eectdotheywanttoachieve,orwhichtaskdotheywantperformed).TheSkill

hierarchyis supposedto beacommongroundwherethecapabilitiesoeredby

devicemanufacturerscanbemappedintotherequirementsoftheusers.Itper-

formedsatisfactorilyduringtheproject,includingthedemonstrationstage,but

ithasbynowbecameclearthatwearemissing acommonunderstandingofan

appropriateabstractionlevelforskills.Forexample,therewasmuchdiscussion

onwhether there should beadrill skill, orifitshould rather be modelled as

a sequence of start drill rotation,move, stop drillrotation operations. In

theend wehavegonewiththeformerapproach,butwearenotconvincedthis

is thebest solution and would rather haveasetup where dierent abstraction

levelscouldbeusedasappropriate.

Therehasbeenmuchdebateabouttheneedforcomplex(orcomposite)skills,

but wedonotthinkitisappropriatetorepresentthemdirectlyin anontology.

It wouldmakemoresense toprovidea moreexpressiveway of buildingthem,

(7)

theirconstraintsandeectsoftheirapplication.

Finally, the Properties hierarchy forms a link between Devices and Skills.

It is, in asense, therst simplest way of specifying the dependencies and

instantiating parameters (for example,that a particular gripperhas a certain

maximumweightlimit).

Throughouttheprojectwehavekeptourinitialassumptionthat allthehi-

erarchieswill be dened by the SIARAS consortium and remain xed, asthe

meaningofalltheconceptsusedneedstobeencodedintothereasoningmecha-

nismsoftheSkillServer.However,ithasbecomeapparentthatthisassumption

isaveryconstrainingone,soitwould begoodto investigatewaysto liftit.

4 Knowledge Representation outside Ontology

The major partof knowledge representation wehave decided to store outside

theontology arethetasks. Taskscanbethoughtofasgeneralisations of skills

along(at least)twoaxes: rst,theyaretime-ordered (orpartially parallelised)

sequencesofskills(orratherofskillapplications):forexample,arelocatetask

canbeseenasasequence of pickup, move and putdown skills. Second,it

makessensetohavehierarchyoftasks,withwindshieldtting taskdecompos-

ingintondwindshield,positionwindshield andx windshield subtasks.

Inaddition,tasksconstitutemeans ofachievingaparticular goal,andSkill

Serverneedstohavearepresentationofthatgoal,inparticulartoreasonabout

rationaleforeachoperationandaboutmotivationswhyaparticulardeviceand

particularparameterswerechosen.Attheveryleastitrequiresasetofcriteria

whichdistinguishacceptableexecutionofthistaskfromunacceptableones,and

possiblywaystocomparetwosolutionsanddeterminewhenoneofthemisbetter

thantheother.

Thebottomleveloftaskshierarchyareoperations,whichareakindofasso-

ciationsof skills, devices,workpieces,factoryoorpositionsand time-line con-

straints.Where tasksareespecially usefulis onahigherlevel,when deninga

concrete productionprocess that will be thesubjectof reconguration.There-

fore the ontologydoes not needthem, orputting it dierently, the only tasks

that areavailableintheontologyarethosethatskillsreferto.

We have chosen Sequential Function Charts[5] to represent tasks, since it

allows us to specify temporal ordering of operations in an intuitive and yet

exibleway.WehaveusedtheopensourcetoolJGrafchartforourprototype.

Finally, we have implemented the core reasoning in Python programming

language, gluing together the knowledge from ontology, SFCs and device-

dependentprocedures providedin theform ofplugins.

Inourapproachtheontologyisusedforreasoningaboutskillsmatchingpar-

ticular tasks (after someinitial re/parametrisation) and devices oering those

skillsundercertainconditions.Apureontologymaybeusedforretrieval,match-

ingandsimpleclassication,whileotherformsofreasoning,likeplanning,opti-

(8)

byin theproject(Racer,Fact++,AlgernonandPellet)dierintheirreasoning

powerand eciency,beingableto handleeither arestrictedDescriptionLogic

language[6](likeOWL-DL oered by Protégé)in an ecient manner[7],ora

fullOWL[8]representation,but usingexponentialsearchalgorithms.Theuser

has the possibility of choosing a dierent reasonerdepending on the question

asked, thus achieving exibility and adaptability of the reasoning partof the

system.

Wehavealsodevelopedtoolsforstoringandretrievingknowledgein appro-

priatedatastructures,sothatononehandtheontologycanbeeasilyextended

bythesystemproviders,whileontheotherhanditmaybenetfromdistribut-

edness,lettingsomepartsbecompletedandstoredatthedevicemanufacturer's

site.Yetanotherset ofrequirementsisput onthereasoningprocessbythelist

of optimisationtasks that mayberequested bythe user.Due to their compu-

tational complexity, and to their specicity to particular devices, they cannot

be implemented in a general-purpose manner but rather require their specic

reasoningblocksttingthestructure oftheserver.

5 Related Work

The research on knowledge representation has been extensively documented,

both in generaltextbooks on articial intelligence and in numerous books de-

votedsolely to this domain. Oneof therecentones,and averygoodoverview

oftheeld,isbyBrachmanandLevesque [9].

Theworkthatoriginateddiscussionsoversemanticweband,inparticular,on

ontologies,hasanextensivelibraryofpublisheddocumentsavailableforexample

attheW3Consortium'sSemanticWebsite[10].Inparticular,thespecications

ofthemostpopularKRformalisms,likeOWL[8]orDAML-OIL[11], together

withavailable toolsforusingthoseformalisms,canbefoundthere.

Production planning is usually considered in AI to be apart of the auto-

maticplanning domain. However,besides theclassicalmanufacturability anal-

ysis,reportedforexamplein [12], thereis in principlenodocumentedresearch

onusingontologiesinautomatedproductionplanning.However,thereisanex-

tensive research aimed at supporting the engineering activities in production

design by providing modelling languagesand tools allowingformal, automatic

analysisofthediscussedprocess.Quitenaturally,mostofthoseformalismsand

toolsareheavily domain-dependent, withasmall numberofexceptionsexplic-

itlystating goalofbeinggeneral-purposetools.WemaynameheretheSensor

ModellingLanguage,orSensorMLforshort,oeringarichsensorontology(see

http://www.sensorml.orgforanextensivedocumentation).Adualenterprise,

not exactlytting our point of vieweither, is the unied robotmodelling lan-

guage, (URML), from theUniversity ofKarlsruhe,asURML doesnot provide

representationfacilitiesforthedynamicaspectsofrobot performance.

Finally,an importantattempt toformalisethelanguageforspeakingabout

(9)

asareferencepointfortheontologydevelopedwithin SIARAS.

6 Conclusions

InthispaperwediscusstheknowledgerepresentationweuseintheEU-funded

project SIARAS, and present the most interesting points where real-life con-

siderationsclashedwithourinitialassumptions. Wehopethisoverviewwill be

helpfultootherpeopledesigningknowledgebasesforindustryapplicationsand

willallowthemtoavoidsomeofthepitfallswehaveencountered.

Themain pointofourdiscussionwastheuseofOntologyand whatarethe

benets and costs associated with representing knowledge in such structured

way. We have discussed situations where ontology turned out to be a rather

cumbersometool,andalsothosewhereitprovedtobeveryhelpful.Wewishwe

wereableto oersomekindof evaluationof theapproachwehavechosenand

oftherepresentationwehaveendedupwith,but unfortunatelywedonothave

anythingmoreconcretethanourownreections.

References

1. Haarslev, V., Möller, R.: Racer: A core inference engine for the semantic web

(2002)

2. Horrocks,I.: FaCTandiFaCT. In:Proceedingsofthe DescriptionLogicsWork-

shop,DL'99.(1999)

3. Stoica, F., Pah, I.: Intelligent agents in ontology-based applications. In: 12th

WSEASInternationalConferenceonComputers.(2008)274279

4. Sirin,E.,Parsia,B.: Pellet:AnOWLDLreasoner.In:DescriptionLogics.Volume

104ofCEURWorkshopProceedings.(2004)

5. Fernández,J.L., Sanz, R.,Domonte, E.P., Alonso,C.: Using hierarchical binary

petrinetstobuildrobustmobilerobotapplications:Robograph. In:International

ConferenceonRoboticsandAutomation.(2008)13721377

6. Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F.,

eds.: TheDescriptionLogicHandbook. CambridgeUniversityPress(2003)

7. Buchheit,M., Donini, F.M., Schaerf, A.: Decidable reasoning in terminological

knowledgerepresentationsystems. JournalofArticialIntelligenceResearch1(1)

(1993)109138

8. W3C: Semanticweb(2003)

9. Brachman,R.J.,Levesque,H.J.: KnowledgeRepresentationandReasoning. Mor-

ganKaufmann(2004)

10. W3C: Semanticweb(2001)

11. Fensel, D., Horrocks, I., vanHarmelen, F., McGuinness, D.L., Patel-Schneider,

P.F.: Oil: An ontology infrastructure for the semantic web. IEEE Intelligent

Systems16(2)(2001)

12. Ghallab, M., Nau, D.,Traverso,P.: Automated Planning,Theory and Practice.

MorganKaufmann(2004)

13. Schleno,C.,Gruninger,M.,Tissot,F.,Valois,J.,Lubell,J.,Lee,J.:Theprocess

specication language (PSL): Overviewand version 1.0 specication. Technical

report,NationalInstituteofStandardsandTechnology(NIST)(2000)