The multi-agent system structure

A Trust Model Based On Communication Capabilities For Physical Agents

Gr´egory Bonnet and Catherine Tessier

ONERA-DCSD, 2 avenue Edouard Belin BP 74025 31055 Toulouse Cedex 4, France

Abstract

In real-world applications where physical agents (such as robots) are used, agents have to share information in order to build a common point of view or a common plan. As agents are generally constrained in their communication capabilities they are likely to decide without consultation. Consequently an agent’s plan may change without the other agents being aware and coordination may be impaired. In such contexts a trust notion about the others’ plans may improve coordina- tion. In this paper we propose a definition of trust based on the agents’ communication capabilities. This model fits co- operative agents under communication and time constraints, such as observation satellites organized in a constellation.

Introduction

In the agent literature, and more precisely in a multi-agent context, most of the coordination mechanisms deal with soft- ware agents or social agents that have high communication and reasoning capabilities. Coordination based on norms (Dignum 1999), contracts (Sandholm 1998) or organizations (Brooks & Durfee 2003; Horling & Lesser 2004) is consid- ered. However some communication is needed in order to share information, trust this information (Josang, Ismail, &

Boyd 2007; Lee & See 2004; Ramchurn, Huynh, & Jennings 2004) and allow agents to reason on common knowledge.

As far as physical agents such as robots or satellites are concerned, the environment has a major impact on coordina- tion due to the physical constraints that weigh on the agents.

Indeed on the one hand, an agent cannot always communi- cate with another agent or the communication possibilites are restricted to short time intervals. On the other hand an agent cannot always wait until the coordination process ter- minates before acting. All these constraints are present in space applications.

Let us consider satellite constellations that are sets of 3 to 20 satellites placed in low orbit around the Earth to take pictures of the ground (Damiani, Verfaillie, & Charmeau 2005). Observation requests are generated asynchronously with various priorities by ground stations or satellites them- selves. As each satellite is equipped with a single observa- tion instrument with use constraints, neighbouring requests Copyright c2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

cannot be realized by the same satellite. Likewise each satel- lite is constrained in memory resources and can realize only a given number of requests before downloading¹. Finally the orbits of the satellites cross around the poles: two (or more) satellites that meet in the polar areas can communicate via InterSatellite Links (ISL) without any ground intervention.

So the satellites can communicate from time to time (a satel- lite meets another one every hour on average). Intuitively intersatellite communication increases the reactivity of the constellation since each satellite is visible by a ground sta- tion (and thus can communicate with it) only 10% of its life cycle (about 10 to 15 years).

In order to coordinate each satellite (each agent) shares its knowledge about tasks and plans with the others when they meet. As the agents are cooperative and honest, their knowledge is trustworthy. But, as communications are de- layed and new tasks may arrive anytime, agents’ knowledge may become obsolete and trust in this knowledge may erode.

Consequently our problem is the following: trust erosion has to be modelled according to the system dynamics so that the agents should plan tasks and coordinate in a relevant man- ner.

First we will present the multi-agent system and the as- sociated communication protocol. According to (Sabater &

Sierra 2005) classification, we will propose a cognitive ap- proach of subjective trust where honest agents propagate di- rect experience and testimony.

The multi-agent system structure

The multi-agent system is defined according to the specifica- tions of our application. It is a satellite constellation defined as follows:

Definition 1 (Constellation) The constellationSis a triplet hA,T,VicinityiwithA = {a1. . . an} the set ofnagents representing thensatellites,T⊂N⁺a set of dates defining a common clock and Vicinity:A×T7→2^Aa symmetric non transitive relation specifying for a given agent and a given date the set of agents with which it can communicate at that date (acquaintance model). Vicinity represents the temporal

1Downloading consists in transferring data to a ground station (i.e. the pictures taken when a task is realized).

windows when the satellites meet; it is calculated from the satellite orbits, which are periodic.

Definition 2 (Periodicity) Let S be a constellation and {p1. . . pn}the set of the orbital cycle durationspi∈Tof agentsai∈A. The Vicinity period˚p∈Tis the lowest com- mon multiple of set{p1. . . pn}.

The constellation (agents, clock and Vicinity) is knowl- edge that all its members hold in common.

Tasks and intentions

In our application each agent knows some tasks to realize and proposes to realize some of these tasks. These proposi- tions are intentions. Tasks can be viewed as objective knowl- edge about the world and intentions can be viewed as judge- ments about these tasks. Consequently intentions may be revised.

Definition 3 (Task) A tasktis an observation request asso- ciated with a priority²prio(t)∈N^∗and to a booleanbtthat indicates whetherthas been realized or not.

When a task is realized by an agent, it is redundant if it has already been realized by an other agent.

Definition 4 (Redundancy) Letaibe an agent that realizes a tasktat timeτ∈T. There is a redundancy about the task tif and only if∃aj ∈ A(ai 6=aj) and∃τ^′ ∈T(τ^′ ≤τ) such thatajhas realizedtat timeτ^′.

One of the goals of the constellation is to prevent redun- dancies via the use of intentions. An intention represents an agent’s attitude towards a given task and the set of an agent’s intentions corresponds to its current plan.

Definition 5 (Intention) LetI_t^aibe the intention of agentai

towards taskt. I_t^ai is a modality of proposition (airealizes t) in terms of:

1. proposal:aiproposes to realizet;

2. withdrawal:aidoes not propose to realizet.

A realization daterea(I_t^ai)∈T∪ {Ø}and a download date tel(I_t^ai)∈T∪ {Ø}are associated with each intention.

The planning process that allows agents to generate in- tentions is beyond the scope of this paper. The mono-agent planning problem may be addressed with many techniques such as constraint programming or HTN planning. As far multi-agent planning is concerned, the problem can be ad- dressed with techniques based on a common knowledge:

coalition formation, contract nets, DEC-MDP’s and so on.

Be that as it may, let us denoteh∈Tan agent’s planning horizon: when agentai plans at timeτ,rea(It^ai)≤τ +h for each generated proposalI_t^ai.

2In the space domain,1stands for the highest priority whereas5 is the lowest. Consequently, the lowerprio(t), the more important tasktis.

meeting

τ a

a

i

j

meeting meeting

a_i

a

a

k

j τ

τi j

Figure 1: Direct and indirect communication

Knowledge

The private knowledge of an agent within the constellation is defined from tasks and intentions:

Definition 6 (Knowledge) A piece of knowledge K_a^τ_i of agentaiat timeτis a triplethDK^τ_ai, AK^τ_ai, τK_ai^τ i:

• DK^τ_ai is a tasktor an intentionI_t^ak ofak aboutt,ak ∈ A;

• AK^τ_ai⊆ Ais the subset of agents knowingK_a^τ_i;

• τK_ai^τ ∈Tis the date whenDK_ai^τ was created or updated.

LetK^τ_ai be the knowledge of agentai at timeτ: K^τ_ai is the set of all the pieces of knowledgeK_a^τ_i.

FromK^τa_i, we define Ta^τ_i ={t1. . . tm} the set of tasks known by agentaiat timeτ; andI_a^τ_i= (I_t^a_j^k)the matrix of the intentions known by agentai at timeτ. Each agentai

has resources available to realize only a subset ofTa^τ_i. Agents communicate this private knowledge in order to build a common knowledge and cooperate.

Communication

Communication is based on Vicinity. When two agents meet they can communicate. Consequently the structure of Vicinity influences the communication capabilities.

Definitions

Two kinds of communications are defined:

Definition 7 (Communication) Let S be a constellation andai,aj be two agentsinS. There are two kinds of com- munications fromaitoaj:

1. Agentai can communicate directly with agentaj iff∃τ

<˚psuch thataj∈Vicinity(ai, τ);

2. Agentaican communicate indirectly with agentaj iff∃ {ak ∈ A, i≤k < j}and∃ {τk <˚p,i≤k < j}such thatak+1∈Vicinity(ak, τk).

As Vicinity is symmetric but not transitive, direct com- munication is symmetric whereas indirect communication is oriented from an agent to another one. Each communication fromai toaj is associated with a couple(τi, τj)∈T²with τithe emitting date ofai andτj the receipt date ofaj. We will write:aicommunicates withajat(τi, τj). In case of a direct communication,τi=τj.

a₁

a

a

2

3

{5;15}

{8;16}

{2;12}

Figure 2: Vicinity graph for Example 1

Unfolding the Vicinity relation

In order to compute the next indirect communication be- tween two agents, Vicinity is projected on an valued- directed-graphV. Formally,

Definition 8 (Multi-valued Vicinity graph) The Vicinity graphVis such thatV = (A,(ai, aj), vij)where:

• Ais the set of vertices ofV;

• the edge(ai, aj)exists iff∃τ,aj∈Vicinity(ai, τ);

• each edge is valued with the setvijof timesτ <˚p.

Let the following example illustrates this definition.

Example 1 Leta1,a2,a3 be three agents. Let us suppose that Vicinity is defined as follows on period˚p= 20:















Vicinity(a1,2) ={a2} Vicinity(a2,5) ={a3} Vicinity(a3,8) ={a1} Vicinity(a1,12) ={a2} Vicinity(a2,15) ={a3} Vicinity(a3,16) ={a1} The vicinity graph is shown on Figure 2.

Intuitively an indirect communication from agent ai to agentaj is a path from vertex ai to vertex aj. Thereby, from this multi-valued graph, we unfold a single-valued graph with respect to the current date and compute the low- est weighted path between both vertices. This single-valued graph is built as it is explored. In order to do that, we pro- pose a modified Dijkstra’s algorithm where:

1. the current timetiis stored in vertexai(initial time plus the weight of the current path);

2. each edge (ai, aj)’s weight is computed as follows:

minvij−ti[ mod ˚p].

Example 2 Let us resume Example 1 and compute from time1the next indirect communication froma1toa3. The weight of edge (a1, a2)is 1 (min(2−1[ mod 20],12−1[

mod 20])). The weight of edge(a1, a₃)is7. Thereby, the current time for vertexa2anda3 are respectively2and8.

A first solution(a1, a3)has been found: a communication at(2,8). Let us continue the exploration from vertexa2. The weight of edge(a2, a3)is3(min(5−2[ mod 20],15−2[

mod 20])) and the current time for vertexa3 is 5. A bet- ter solution has been found: an indirect communication at (2,5).

An epidemic protocol

In our application, we consider an epidemic protocol based on overhearing (Bonnet & Tessier 2007; Legras & Tessier 2003). The main idea is to take advantage of every com- munication opportunity, even when an agent communicates information that does not concern itself:

1. each agentaiconsiders its own knowledge changes;

2. aipropagates the changes toaj ∈Vicinity(ai, τ);

3. ajupdates its knowledge thanks to the timestampτK_ai^τ . It has been proved that, in a set ofnagents where a single agent knows a piece of information, an epidemic protocol needsO(logn)communication rounds to completely prop- agate this information (Pittel 1987).

Common knowledge

The agents have to reason on a common knowledge in terms of tasks and intentions. Because of the communication de- lays, this common knowledge concerns only a subset of agents. Formally,

Definition 9 (Common knowledge) At time τ, agent ai

knows that agentajknows the intentionI_t^aicaptured byK_a^τ_i iff:• aj∈AK_ai^τ or

• ai communicated withaj at(τi, τj)such thatτK^τ_ai ≤τi

andτj≤τ.

The trust model

Our application can be viewed as an ad-hoc network. How- ever trust literature on ad-hoc networks (Pirzada & McDon- ald 2006; Theodorakopoulos & Baras 2006; Virendra et al.

2004) focus on the reliability of a node in itself and the way to route reliable information. In our application, as agents are trustworthy, trust erosion does not come from the nodes themselves but from interactions between nodes.

Trust in the last communication

When two agents communicate at timeτ, the agent that re- ceives a given intention cannot be sure that this intention will be the same at timeτ^′ (τ^′ > τ). Indeed as the envi- ronment is dynamic, an agent may receive new tasks or new intentions and modify its plan, i.e. its own intentions, ac- cordingly. The more time between the generation of a given intention and the realization date, the less an agent can trust this intention. However a further confirmation transmitted by the agent that has generated this intention increases the associated trust again.

As we consider the agents honest and cooperative, an in- direct communication (which is a testimony) is trustworthy in itself. Thereby an agentaiconsiders that a given proposal generated by an agentajhas been confirmed ifajcommu- nicates (directly or not) withai without modifying its pro- posal. We define formally the last confirmation.

Definition 10 (Last confirmation) Letaibe an agent,I_t^aj a proposal of an agentaj about a tasktknown byai. The last confirmation of proposalI_t^aj foraiat timeτis:

τ^∗= max

τ_Kτ

ai<τ_j τi<τ

{τj :aj communicates withai at(τj, τi)}

Example 3 Let us resume Example 1. Let us suppose that, at time15,a3computes the trust associated with an inten- tion of agenta1generated at time7. a1communicated di- rectly witha3at time8then it communicated indirectly with a3at time(12,15)without modifying its proposal. Thereby the last confirmation is12anda3knows thata1kept its pro- posal between times7and12.

Trust computation

Intuitively the trust associated with a proposal depends on the time between its last confirmation and its realization. As the agents do not have a model of the environment, they can- not predict the arrival of new tasks. However the more time passes, the more an agent meets other agents. Each meeting is an opportunity for an agent to receive new tasks and to revise its intentions.

Consequently the trust about a given proposal is defined from the number of meetings between the last confirmation and the realization date. This number is based on Vicinity, therefore each agent can compute its own trust in the others’

intentions.

Definition 11 (Meetings) Let ai be an agent, I_t^aj a pro- posal known byaiandτ the current date. Letτ^∗be the last confirmation ofI_t^aj foraiat timeτ. The number of agents M_τ^a∗ⁱ(I_t^aj)agentaj will meet betweenτ^∗ andrea(I_t^aj)is M_τ^a∗ⁱ(It^aj) =| [

τ^∗<τ^′<rea(I_t^aj)

Vicinity(aj, τ^′)|

Finally an agent trusts or does not trust a given proposal.

Let us define the conditions of trust for an agent:

Definition 12 (Trust) Let ai be an agent, I_t^aj a proposal known byaiandτthe current date. Agentaitrusts agentaj

aboutI_t^aj if and only ifM_τ^a∗ⁱ(It^aj) = 0.

The following example illustrates our proposition.

Example 4 Letai be an agent that knows proposalI_t^aj at timeτ. Let us suppose thatM_τ^a∗ⁱ(I_t^aj) = 5. Agentaidoes not trustaj about this proposal. Let us suppose thataj keeps its intention for long enough to confirm it twice. At each confirmation,aican computeM_τ^a∗ⁱ(I_t^aj)again, respectively 3and1. Each time,ai trustsaj more. Figure 3 shows this evolution.

We can notice that the trust criterion is hard: an agent is not trusted if it meets at least another agent before realizing its intention (M_τ^a∗ⁱ(I_t^ak) = 0). This pessimistic assumption can be relaxed (e.g.M_τ^a∗ⁱ(It^ak)≤1).

Given the structure of Vicinity, it exists a number of meet- ings sufficient to always trust a proposal.

Definition 13 Let S be a constellation and h ∈ T the agents’ planning horizon. ∀ai,aj ∈ A, let us denotef_a^a_jⁱ the maximal number of meetings between two direct com- munication betweenai andaj in each intervalh <˚p. The maximal trust threshold is:

a_j

τ

time

2 meetings 2 meetings 1 meeting confirmation confirmation trust criterion

rea(I ) t

Figure 3: Trust evolution for Example 4

F = max

a_i,a_j∈Af_a^a_jⁱ

Example 5 Let us resume Example 1. At most,h = ˚pand each agent ai can meet another agent before meeting an agentaj. Thereby, the maximal trust threshold is1. Two behaviours can be considered:

1. a pessimistic one: an agent distrusts another agent as soon as it meets a third agent before realizing its inten- tions;

2. an optimistic one: an agent always trusts the others.

Experiments

The model we have described has been implemented. Expe- riments have been conducted on three kinds of agents:

• optimistic agents: an agent always trusts the others;

• cautious agents: an agent trusts another agent if it meets at most a third agent before realizing its intentions;

• pessimistic agents: an agent distrusts another agent as soon as it meets a third agent before realizing its inten- tions.

The experiments are based on a scenario with3 agents, 50 tasks and a task density parameter. We have launched 100simulations and computed the average result, i.e. the number of realized tasks without any redundancy. In these simulations, there is no arrival of new tasks.

Definition 14 (Density) The task density represents how close to each other the tasks are. The closer the tasks, the more they are likely to be in mutual exclusion.

During the simulations, the agents coordinate a priori and use common knowledge to reduce redundancies. When an agent detects a possible redundancy between its own propos- als and another agent’s proposals, it maintains its proposals if the other agent is not trusted. Thereby the untrusted pro- posals provoke redundancies. A commitment mechanism based on coalition formation has been defined in (Bonnet &

Tessier 2008) to avoid both agents to withdraw. The results are given in Figure 4.

The differences between the different kinds of agents rep- resent the untrusted intentions that produce redundancies

100 % 80 %

60 % 40 % 20 %

Tasks without redundancy

Tasks density 10

20 40

30 50

cautious agents pessimistic agents optimistic agents

Figure 4: Trust effects on coordination

and thereby the amount of backup tasks in case of intention modifications. When task density is low, optimistic agents outperform the other kinds of agents ; but as the density in- creases, the benefit provided by optimistic agents decreases quicker than the benefit provided by other kinds of agents.

Conclusion

We have proposed a trust model to deal with common knowledge in a cooperative multi-agent system. This kind of approach is well-suited for real-world applications such as the coordination of satellite constellations.

The main ideas are the followings. The agents must share knowledge about tasks and intentions about tasks. As new tasks may appear in the system, the agents may revise their plans, that is to say their intentions. However in order to co- ordinate, the agents must rely on the other’s intentions: they must trust them. Thereby we propose a trust notion which is defined through the communications between agents. Each time an agent communicates, it may receive new informa- tion that modifies its intentions; on the other hand the more an agent communicates, the more it can confirm its inten- tions and the more trust may increase.

Some experiments have been conducted where we have investigated the effect of the task density on trust. Of course, the more the agents trust the others, the more they cooper- ate. Conversely the less the agents trust the others, the more they keep redundant intentions. These redundant intentions are backups in case of failure or high dynamics (new tasks arrivals). Finally in case of honest and cooperative agents, the choice of the trust threshold is one of the key parameters of the coordination mechanism.

References

Bonnet, G., and Tessier, C. 2007. Collaboration among a satellite swarm. In Proceedings of the 6th AAMAS, 287–294.

Bonnet, G., and Tessier, C. 2008. Incremental adaptive organi- zation for a satellite constellation. In Proceedings of the OAMAS workshop at the 7th AAMAS.

Brooks, C., and Durfee, E. 2003. Congregation formation in mul- tiagent systems. Journal of Autonomous Agents and Multiagent Systems Vol. 7:145–170.

Damiani, S.; Verfaillie, G.; and Charmeau, M.-C. 2005. An Earth watching satellite constellation : How to manage a team of watch- ing agents with limited communications. In Proceedings of the 4th AAMAS, 455–462.

Dignum, F. 1999. Autonomous agents with norms. Artificial Intelligence and Law Vol. 7:69–79.

Horling, B., and Lesser, V. 2004. A survey of multi-agent orga- nizational paradigms. The Knowledge Engineering Review Vol.

19:281–316.

Josang, A.; Ismail, R.; and Boyd, C. 2007. A survey of trust and reputation systems for online service proposition. Decision Support Systems Vol. 43(2):618–644.

Lee, J., and See, K. 2004. Trust in automation: Designing for appropriate reliance. Human Factor Vol. 46(1):50–80.

Legras, F., and Tessier, C. 2003. LOTTO: group formation by overhearing in large teams. In Proceedings of 2nd AAMAS.

Pirzada, A. A., and McDonald, C. 2006. Trust establishment in pure ad-hoc networks. Wireless Personal Communications Vol.

37(1-2):139–168.

Pittel, B. 1987. On spreading a rumor. SIAM Journal of Applied Mathematics Vol. 47:213–223.

Ramchurn, S. D.; Huynh, D.; and Jennings, N. R. 2004. Trust in multi-agent systems. The Knowledge Engineering Review Vol.

19(1):1–25.

Sabater, J., and Sierra, C. 2005. Review on computational trust and reputation models. Artificial Intelligence Vol. 24(1):33–60.

Sandholm, T. 1998. Contract types for satisficing task alloca- tion. In Proceedings of the AAAI Spring Symposium: Satisficing Models, 23–25.

Theodorakopoulos, G., and Baras, J. S. 2006. On trust models and trust evaluation metrics for ad-hoc networks. IEEE Journal on Selected Areas in Communications Vol. 24(2):318–328.

Virendra, M.; Jadliwala, M.; Chandrasekaran, M.; and Upad- hyaya, S. 2004. Quantifying trust in mobile ad-hoc networks.

In Proceedings of the IEEE International Conference on Integra- tion of Knowledge Intensive Multi-Agent Systems, 65–71.