Action and Goal Recognition

Action recognition is the problem of observing the behavior of an agent and inter-preting that behavior as a discrete action or actions. Related to this notion is the more well-known articial intelligence problem of goal or plan recognition, which is concerned with explaining the behavior of an agent¹. The two notions are distinct, however, because research on the former topic focuses on the issue ofwhatan agent is doing, whereas the latter topic concentrates onwhyan agent is doing what it is doing.

These issues are of interest for cooperative mobile robotics because they can provide robots with the ability to respond more appropriately to the actions and intentions of their teammates. Without at least a rudimentary ability to perform action and goal recognition, robot teams will have diculty achieving coherence in their task selec-tion, as discussed in the following section. In this secselec-tion, I explore the issues of goal recognition and action recognition, reviewing the related research in these areas and making comparisons to the approach employed in ALLIANCE and L-ALLIANCE.

5.1.1 Goal Recognition

A signicant amount of research in articial intelligence has addressed the topic of goal recognition. An application domain particularly well-studied is that of natural language discourse understanding. This research deals primarily with the role of plans and intentions in understanding dialogues. A broad selection of papers in this area can be found in [Cohen et al., 1990b]; additional work in this area is described in [Carberry, 1990, Charniak and Goldman, 1989, Konolige and Pollack, 1989, Mayeld, 1989]. A second main body of work in goal recognition is the area of intelligent user interfaces; examples of this area of research can be found in [Goodman and Litman, 1990, Raskutti and Zukerman, 1991].

Although the specic approaches to goal recognition vary greatly, nearly all ap-proaches involve the use of a model of agent behavior for use in interpreting that

1Since the meanings of goal recognition andplan recognition are often blurred in AI and Dis-tributed AI research, I henceforth use the term goal recognition to encompass bothgoal and plan recognition as used by the traditional AI and DAI literature. I do this more for philosophical reasons than for conciseness, because the termplanrecognition implies that the observed agent actuallyhas a plan, in the traditional AI sense (see chapter 8 for a description of what is meant by \traditional AI"). I resist making this assumption as applied to mobile robots for reasons that should be obvious from this report.

5.1. ACTIONANDGOAL RECOGNITION 129 agent's actions. Bond and Gasser in [Bond and Gasser, 1988] cite the reasons for modeling agents as follows:

Models are useful for predicting the requirements and eects of events not di-rectly sensible (e.g. because they will occur in the future).

Models can reduce communications requirements.

Models can be useful for evaluating the credibility, usefulness, reliability, or timeliness of data.

Models may improve eciency by focusing activity or by directing search.

It is interesting to note that most of the agent models proposed in this body of research are quite elaborate, and stress the manipulation of declarative (as opposed to procedural) knowledge of agent actions. The usefulness of these elaborate models for goal recognition within physical robots that must operate in real-time, however, is uncertain. As described in more detail in chapter 8, this type of elaborate modeling used by traditional articial intelligence approaches for general robot control has not performed well when applied to physical mobile robots. In fact, the most successful mobile robots to date have been built according to lessons learned from ethology, in which a few relatively simple rules of action interact to create the emergent behavior of the robot [Maes, 1990]. Thus, we might expect that the same principle which holds for general robot control also holds for goal recognition in physical robots.

Studies involving East African vervet monkeys [Cheney and Seyfarth, 1990] have indeed shown that these animals view the world as things that act, not as things that think andfeel. In other words, these monkeys can well understand behaviors of other animals in their society without having a concept of the knowledge or beliefs that may have caused those behaviors. Even without the ability to model the beliefs of other monkeys, however, these animals are able to cooperate to an amazing extent.

Thus, we have an existence proof that complex models of intention are unnecessary for cooperation at the level exhibited by most social animals. Indeed, I have shown in this report that for the application domains I have studied, elaborate understanding of robot intentions is not necessary to achieve cooperative control.

Agent modeling issues aside, as noted in [Huber and Durfee, 1993], the existing goal recognition research is of limited use in multi-robot cooperation because the research almost invariably assumes that error-free symbolic descriptions of the current action (and possibly previous actions) taken by an agent and the current state of the world are always available. Since the systems do not deal with the dicult problem of obtaining the symbolic descriptions in the rst place, nor with the problem of uncertainty in the observations, they have little relevance to physical robot domains.

One exception to this is the preliminary work described in [Huber and Durfee, 1993], which explicitly addresses this issue for mobile robot cooperation. In this paper, Huber and Durfee describe experiments in which one robot tries to infer the goal destination of another robot by analyzing its movements and taking into account uncertainties in the observations.

5.1.2 Action Recognition

Recognizing the actions of teammates can be performed by a robot in one of two ways:

Through explicit communication

Through interpretation of behavior observations

Clearly, the easiest method is the rst, which involves having each robot explicitly communicate its current action to teammates according to some protocol or pre-determined arrangement. As we have seen, this is the method utilized in ALLIANCE and L-ALLIANCE, whereby robots broadcast their current actions to teammates at some specied rate.

However, this simple method will obviously not work for applications in which the communication medium is not available (e.g. due to a noisy environment or faulty equipment), is costly (e.g. in terms of time or robot safety, in military applications), or is of limited bandwidth. In such applications, the robots must rely on other sensors

| primarily visual | to observe and interpret the actions of team members.

Unfortunately, research in action recognition is much more limited than that ad-dressing goal recognition, due to the diculty of passive action interpretation. The current state of the art in this eld provides robots with the ability to interpret simple teleoperated assembly tasks using non-visual sensors [Takahashi et al., 1993, Yanget al., 1993] and the ability to visually recognize human actions in simple blocks-world construction tasks [Kuniyoshi and Inoue, 1993, Ikeuchiet al., 1993]. All four of the projects cited involve a robot rst observing the human performance of a task and then interpreting and converting that task into either a symbolic or a kinematic representation which can then be reproduced by the robot. These research projects are notable for advancing the the eld of passive action interpretation. However, they also demonstrate the signicant diculty of the action recognition problem, since they currently only work for very simple problems.