Compositional Verification of Agents in Dynamic Environments: a Case Study
Catholijn M. Jonker, Jan Treur, Wieke de Vries
Vrije Universiteit Amsterdam
Department of Mathematics and Computer Science, Artificial Intelligence Group De Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands
URL: http://www.cs.vu.nl/~{jonker,treur,wieke} Email: {jonker,treur,wieke}@cs.vu.nl
Abstract. In this paper compositional verification of agents in dynamic environments is studied. Dynamic properties of an example agent in a dynamic environment are identified in relation to the different abstraction levels of the compositional structure of the system. The properties are formalised using temporal models. Mathematical proofs relate the properties at the different process abstraction levels. The dynamics of the environment has several consequences for the verification process. Properties often have to contain conditions concerning the dynamic behaviour of the world. In the proofs, the partly unpredictable behaviour of the word has to be taken into account.
This complicates the verification process. A number of aspects of proof pragmatics (i.e., heuristics for finding proofs) identified during this analysis and aimed at controlling the proof complexity, are discussed.
1 Introduction
With the increase of the complexity of systems and the sensitivity of those systems with respect to security, safety, and costs, the need for verification becomes more important every day. The purpose of verification is to prove that, under a certain set of assumptions, a system will adhere to a certain set of properties, for example the design requirements; see also, e.g., (Fensel, 1995; Fensel and Benjamins, 1996; Fensel, Schonegge, Groenboom, and Wielinga, 1996; Harmelen and Teije, 1997). In our approach, a mathematical proof (i.e., a proof in the form mathematicians are accustomed to do) is given that the (detailed) specification of the system together with the assumptions implies the properties that it needs to fulfil. In this sense verification leads to a formal analysis of relations between properties and assumptions.
Given the increasing complexity of systems to be verified, the need for a systematic approach to verification that leads to a comprehensible proof is paramount. The first experiences with the compositional verification method introduced in (Cornelissen, Jonker, and Treur, 1997) for knowledge-based systems with a case study in diagnostic reasoning, and in (Jonker and Treur, 1998a) for multi-agent systems applied to a case study in reactiveness and pro- activeness of agents acquiring information about a static world, were very promising. The proofs are structured in comprehensible manner and constructed by making use of the compositional structure of the system being verified.
Although the systems verified in these first attempts are reasonably complex, the material (or external) world considered is static. In this paper the compositional verification method is applied to formally analyse an agent system with a dynamic world.
In the verification process again the different abstraction levels of the compositional structure of the system were used to identify dynamic properties of the agent, of the external world and of the interaction between them.
Mathematical proofs relate the properties at the different
process abstraction levels. This mathematical style is used because it is the most general way to formalise and verify system behaviour; it provides maximal expressive freedom.
Using a formal logic, with a limited number of operators, could be too constraining in the present phase of the research. But in the future, a formal logic will be chosen to conduct the verification. To explore demands on this logic, the mathematical proof style is useful. The properties are formalised using temporal models with incomplete states (to express ignorance). The dynamics of the external world proved to make verification more complex; more effort was needed to create comprehensible proofs. Valuable experience was gained during this analysis that resulted in a number of basic assumptions and workable heuristics for finding proofs.
This case study is used to identify possibilities as well as problems that are encountered in the verification of multi-agent systems in dynamic environments. The properties verified in this paper are relevant for multi-agent systems in dynamic environments, although their formalization is domain-dependent. For other agent systems in dynamic worlds, the properties only have to be adjusted to the particular application domain. A general strategy is used to obtain the properties and to prove them. Genericity and reusability are of major importance. The aim of the research is to find a general approach of verification of multi-agent systems that are developed in a compositional and conceptual manner. To obtain this approach, a series of case studies is being performed, including the one described in this paper. The approach should identify common aspects of verification of all kinds of multi-agent systems.
For example, there are types of properties that play a role in many multi-agent systems. Also, general applicable heuristics to reduce the search space of the proofs to a manageable size have been found (and more will have to be found). Using this approach of multi-agent system verification, the verification process will become more structured and algorithmic in nature. Tools to execute and/or aid with the verification could then be created.
2
Section 2 contains an overview of the compositional verification approach, which constitutes the foundation of the sought-for verification approach. In Section 3 a problem description (a description of a pseudo-experiment) of animal behaviour is presented, the basic requirements are formulated, and the model is presented, that is to be verified in the subsequent sections. Section 4 contains the top-level properties of the system, that are proven in Section 5, using assumptions on the behaviour of the dynamic world.
Some of these assumptions appear in Section 4. Section 6 discusses basic assumptions that play a central role. In Section 7 proof heuristics that (may) play a role in the verification of dynamic systems are identified. Section 8 discusses the interaction between design processes and verification processes. Conclusions and further perspectives are discussed in Section 9.
2 Compositional Verification of Dynamic Properties
The complexity of the verification process is one of the major concerns in verification of non-trivial systems. In particular, for verification of dynamic properties of a system, a huge search space has to be faced. Compositional verification is an approach meant to handle this complexity, by structuring proofs according to different process abstraction levels; e.g., (Cornelissen, Jonker and Treur, 1997; Jonker and Treur, 1998a); see also (Abadi and Lamport, 1993; Hooman, 1994; Dams, Gerth and Kelb, 1996). A compositional multi-agent system can be viewed at different levels of process abstraction. Viewed from the top level, the complete system is one process (modelled by a component S), with interfaces, whereas internal information and processes are hidden (information and process hiding). At the next lower level of abstraction, the system component S can be viewed as a composition of agents and the world, and information links between them.
Each agent is composed of its sub-components, and so on.
Compositional verification takes this compositional structure into account: it plays a heuristic role in finding the properties and proofs.
2.1 Verification and Levels of Process Abstraction
Often the properties that need to be verified are not given at the start of the verification process. Actually, the process of verification has two main aims:
• to find the properties
• given the properties, to prove the properties The verification proofs that connect one process abstraction level with the other are compositional in the following manner: any proof relating level i to level i+1 can be combined with any proof relating level i-1 to level i, as long as the same properties at level i are involved. This means, for example, that the whole compositional structure beneath level i can be replaced by a completely different design as long as the same properties at level i are achieved.
After such a modification the proof from level i to level
i+1 can be reused; only the proof from level i-1 to level i has to be adapted. In this sense the verification method supports reuse of verification proofs.
2.2 The Temporal Semantics Used
In principle, verification is always relative to semantics of the system descriptions that are verified. For our Compositional Verification approach, these semantics are based on compositional information states which evolve over time. In this subsection a brief overview of these assumed semantics is given.
An information state M of a component D is an assignment of truth values {true, false, unknown} to the set of ground atoms that play a role within D. The compositional structure of D is reflected in the structure of the information state. A formal definition can be found in (Brazier, Treur, Wijngaards and Willems, 1996). The set of all possible information states of D is denoted by IS(D).
A trace of a component D is a collection of information states (Mt)t T in IS(D) over a time structure T. For this paper T will be chosen as a dense ordering, e.g., the non-negative real numbers. The set of all traces is denoted by IS(D)T, or Traces(D). If C is a sub-component (or sub-sub-component, or ...) of D, by Traces(D)|C the restriction of the traces to C is meant, that is, only that part of each information state that pertains to C is considered.
Given a trace of component C, the information state of the input interface of component C' at time point t is denoted by state( , t, input(C')), where C' is either C or a sub-component of C, a sub-sub-component of C, etc.
Analogously, state( , t, output(C')), denotes the information state of the output interface of component C' at time point
t.
3 Problem Description
One of the most important aspects of agents (cf.
(Wooldridge and Jennings, 1995)) is their behaviour. In the past, behaviour has been studied in different disciplines. In Cognitive Psychology the analysis of human behaviour is a major topic. In Biology, animal behaviour has been and is being studied extensively. In one approach animal behaviour is explained only in terms of a black box that for each pattern of stimuli (input of the black box) from the environment generates a response (output of the black box), that functionally depends on the input pattern of stimuli;
i.e., if two patterns of stimuli are offered, then the same behaviour occurs if the two patterns of stimuli are equal. In this section a generic model of a purely reactive agent is briefly presented which is an adequate agent model to describe the (immediate) functional character of stimulus- response behaviour (Jonker and Treur, 1998b). The black box is represented by the agent component. The stimuli form the input (observation results), and the response is formed by the actions to be performed which are generated as output.
In this article a concrete example domain is considered that is taken from the discipline that studies animal behaviour; see e.g., (Vauclair, 1996).
3.1 The Domain
One type of experiment reported in (Vauclair, 1996) is set up as follows (see Figure 1). Separated by a transparent screen (a window, at position p0), at each of two positions
p1 and p2 a cup (upside down) and/or a piece of food can be placed. At some moment (with variable delay) the screen is raised, and the animal is free to go to any position.
Consider the following three possible situations:
Situation 1 At both positions p1 and p2 an empty cup is placed.
Situation 2 At position p2 a piece of food is placed, which is (and remains) visible for the animal. At position p1 there is nothing.
Situation 3 At position p1 an empty cup is placed and at position p2 a piece of food is placed, after which a cup is placed at the same position, covering the food. After the food disappears under the cup it cannot be sensed anymore by the animal.
In situation 1 the animal will not show a preference for either position p1 or p2; it may even go elsewhere or stay where it is. In situation 2 the animal will go to position
p2, which can be explained as pure stimulus-response behaviour. In situation 3 the immediate stimuli are the same as in situation 1. Animals that react in a strictly functional stimulus-response manner will respond to this situation as in situation 1. Models of animals that show delayed response behaviour (and will go to p2, where food can be found) or other types of behaviour can be found in (Jonker and Treur, 1998b).
p0
p1
p2
Fig. 1. Situation 3 of the experiment
3.2 The Requirements
In this paper a purely reactive agent model is described for the experiment. The following requirements on its behaviour are formulated: The agent should behave the same for the situations 1 and 3 described above: doing nothing, as if no food is present. Only in situation 2 should it go to the position of the food and eat it.
3.3 An Agent Model for Purely Reactive Behaviour
For the design and implementation of the different models the compositional development method for multi-agent systems DESIRE has been used; see (Brazier, Dunin- Keplicz, Jennings, and Treur, 1997) for more details. A generic agent model for purely reactive behaviour developed earlier within the DESIRE environment (and applied in chemical process control) was reused.
3.3.1 Process Composition
The (rather simple) agent system (denoted by S) consists of two components, one for the agent (denoted by M) and one for the external world (denoted by EW) with which it interacts (see Figure 2).
In the current domain, the observation information that plays a role describes that certain objects (cup1, cup2, food,
screen, mouse, self) are at certain positions (i.e., p0, p1, p2).
This is modelled by two sorts OBJECT and POSITION and a relation at_position between these two sorts. Moreover, two types of actions can be distinguished: eat and goto some position. The latter type of actions is parameterized by positions; this can be modelled by a function goto from
POSITION to ACTION. E.g., goto(p1) is the action to go to position p1. The action eat that is specified assumes that if the animal is at the position of the food, it can have the food: if a cup is covering the food, as part of the action eat
the animal can throw the cup aside to get the food.
Variables over a sort, e.g., POSITION, are denoted by a string, e.g., P, followed by : POSITION, i.e., P : POSITION is a variable over the sort POSITION. The unary relation
to_be_performed is used to express the information that the agent has decided to perform an action; for example,
to_be_performed(goto(p1)) expresses that the agent has decided to go to position p1. The relation observation_result is used to express the meta-information that certain information has been acquired by observation; for example,
observation_result(at_position(food, p1), pos) expresses that the agent has observed that there is food at position p1, whereas the statement observation_result(at_position(food, p1), neg)
expresses that the agent has observed that there is no food at position p1.
agent external
world top level
observation results actions and observations
Fig. 2. A generic agent model for purely reactive behaviour
4
Within the process composition so far, the external world has been treated as a black box. This has the advantage that the system can easily be adapted to function either with a simulated world or with the real world. This approach is valuable for verification as the system can be verified up to the external world; leaving the external world as a black box assumed to satisfy certain properties. For a simulated external world the verification process can continue to be certain that the simulation has the required properties.
3.3.2 The Domain Knowledge
Assuming that food is offered at most one position (for example, position p2), the stimulus-response behaviour of agent model A expresses that if the agent observes that there is food at any position and that no screen at position
p0 separates the agent from this position, then it goes to this position. Also, if the agent observes that it is at the position of the food, the agent decides to eat the food. This knowledge has been modelled in the following form:
if observation_result(at_position(food, P:POSITION), pos) and observation_result(at_position(screen, p0), neg) and observation_result(at_position(self, P:POSITION), neg) then to_be_performed(goto(P:POSITION))
if observation_result(at_position(self, P:POSITION), pos) and observation_result(at_position(food, P:POSITION), pos) then to_be_performed(eat)
4 Different Types of Properties
As an example of our verification method for dynamic systems the behaviour of the system presented in Section 3 is to be verified for situations in which the food is visible at one of the positions; the proof obligation is that the food will disappear after the screen is gone (called screenrise).
This is formalised in the following property:
S0. The food has disappeared some time after screenrise:
!
" # !
$$% $$
&
' $( $ " #
A heuristic to prove properties like this one is to make use of a combination of top-down and bottom-up approaches.
Top-down: With this property in mind formulate properties for the sub-components (behavioural properties) and for the co-operation between those sub-components (environment and interaction properties) that might be useful. Formalise them, and then use a bottom-up strategy to see whether or not these properties are enough to prove the main property.
In this case, properties of the agent, of the world, of the co- operation between agent and world are in order.
4.1 Agent Properties
Property S0 is phrased in terms of the component EW, that is responsible for the maintenance of the correct state of the world. Therefore, the property describes correct behaviour purely in terms of world situations. But to obtain this behaviour, the agent component has to make the right
decisions. Because S0 depends on correct agent behaviour and not just on correct world behaviour, it is a property of the entire system S. The correct reasoning of the agent is described with four properties.
M1. Effective moving decision making of M: decisions of M to move are made if the circumstances are observed to be appropriate.
)
)
* + , - !
* + , " # !
* + , ," -
&
' $ ( $ ) * ". # -
This property states that the agent decides to perform a goto-action when it observes that the screen is gone and that there is food at a position different from its own.
M2. Justified moving decisions of M: decisions of M to move are only made if the circumstances are observed to be appropriate.
)
) * " . # -
&
' $% $ )
* + , - !
* + , " # !
* + , ," -
This property states that the agent only decides to move when there are good reasons, these being the absence of the screen and the presence of the food somewhere else.
M3. Effective eating decision making of M: decisions of M to eat are made if the circumstances are observed to be appropriate.
)
)
* + , " # !
* + , ,"
&
' $( $ ) * " . #
The above property formalises that the agent decides to eat when it observes that it is with the food.
M4. Justified eating decisions of M: decisions of M to eat are only made if the circumstances are observed to be appropriate.
) '
) * ". #
&
' $% $ )
* + , " # !
* + , ,"
This last agent-property states that the agent only decides to eat when it observes that it is with the food.
The agent component M is a primitive component;
these properties can be proven directly from the knowledge base of the agent component (see Section 3.3.2), without using other properties. Such properties are called basic properties. As can be seen by comparing the knowledge base of the agent with the properties, the properties formalise the correct functioning of the rules of the knowledge base in the reasoning process. For each rule, there is a property stating that the conclusion will be drawn
if the premises hold, and one property stating that the conclusion will be drawn only if the premises hold.
In general, properties formalising the correct functioning of primitive components of arbitrary systems are very similar to the above properties.
4.2 Interaction Properties
The agent and the external world are connected through two information links, that transfer observation results from the world and actions for the world, respectively. Properties of information exchange have a general format, valid for every system containing links. There are two kinds of properties.
Interaction effectiveness states that information from the source of a link is correctly delivered at the destination of the link some time later, and interaction groundedness states that when particular information is present in an interface, corresponding information must have been present in the source of one or more links some time earlier. One example of information exchange will be described, namely the information flow to the external world, with its two properties. Other interaction properties are analogous.
I1. Interaction effectiveness from M to EW:
/ - # , , "
) / &
' $( $ /
I2. Interaction groundedness of input information of EW:
/ - # , , "
/ &
' $% $ ) /
Interaction properties are also basic properties.
4.3 System Properties
Some properties are needed that seem to concern one component, however, in reality depend not only of the behaviour of that component, but also on the behaviour of the links and components that interact with it (this is called its environment). These properties are properties of the whole system S.
S 9 . When the environment of EW is provided with the necessary observation results, it provides a goto-action:
* + , - !
* + , " # !
* + , . -
&
' $ ( $ * " . # -
S10. The environment of EW only provides a goto-action when the necessary observation results were present:
* " . # -
&
' $% $
* + , - !
* + , " # !
* + , . -
S11. When the environment of EW is provided with the necessary observation results, it provides an eat-action:
* + , " # !
* + , .
&
' $( $ * " . #
S12. The environment of EW only provides an eat-action when the necessary observation results were present:
'
* " . #
&
' $ % $
* + , " # !
* + , .
4.4 Properties of the External World
Even though the world is dynamic, this doesn’t mean that everything is possible. Strange events should not occur: for example, the agent never ever disappears and the agent only moves according to the actions decided upon by the agent.
These properties are formalised as follows:
W17. The mouse is always somewhere:
' 0 1 2 21 3
.
W26. When the mouse changes position, there must have been a goto-action on the input of EW:
0 1 2 21 3 4 5 0 1 2 21 3
. 4 !
' $% $ . &
' $$% $$ * " . # - 4
Furthermore, the observation results provided by the external world should correspond to the current world state.
W21. Observations from EW were facts:
6 - # . 7 ,# "
* + ,6
&
' $% $ 6 !
* + ,6 -
&
' $% $ 6
Finally, some assumptions on the behaviour of the external world are necessary. Since, the goal is to prove the system correct for situation 2, the external world is assumed to behave according to situation 2, i.e., food does not move around, food does not disappear unless it is eaten, food remains visible at all times and the mouse is initially at p0. This last property is given:
W18. Initially, the mouse is at p0:
. !
. !
.
Properties of the external world are not influenced by the agent. In the verification process they are used as assumptions to prove the system properties. If in the system design the external world is the real physical world, the properties have to be obeyed by the world in order to obtain the desired system behaviour. In case of a simulated external world, containing several sub-components, the properties can be proven from properties at a deeper level of abstraction.
