Dealing with low-level processing Noise

One of the major challenges to deal with, is the management of noise from low-level pro-cessing . This noise, is one of the first source of uncertainty in the event recognition process. In this section, we present the proposed strategies to take into account the uncertainty inherent in low level observations.

3.6.3.1 Visual Reliability

One of the first notion in the litterature to qualify uncertainty is the notion of reliability.

In this thesis, we define the notion of ‘visual reliability’. The visual reliability is intended to quantify how much the object can be seen from the camera point of view. The objective is to find a measure that gives a minimal value when the object is not visible, and a maximal value when the object is totally visible. We define the visual reliability for 2D and 3D attributes.

3.6.3.2 Dynamic Model for temporal attribute filtering

Observations in real world video sequences can be corrupted by noise, thus our goal is to estimate more accurately an attribute value given its observed value. We propose a dynamic linear model for reliability computing and updating the attributes valueaand confidence based on a temporal history of the previous values (see chapter 5 for more details). We think that tacking into account an history of the previous values and not just the previous one help us to a better estimation. This process which follows the same strategy than a kalman filter works in two steps:

-The first step (1)consists in computing the expected valuea_expof an attributeaat the current instantt_cgiven the estimated value ofaand its velocity at the previous timet_p.

-The second step (2)is to compute the estimated valueaestof the attribute based on the previous one.

Figure3.9: An utility coefficient is associated to each sub-event of the event model. The primitive state

‘close-to’ is associated with an utility equal to 1, that is mean that this primitive state is highly required to recognize the composite state ‘Person-interacts-with-chair’. The primitive state ‘inside-zone’is associated with an utility equal to 0.8.

-The final valuea¯ of the attribute is the mean between the expected and the estimated values of the attribute weighted by the expected and estimated reliability values

3.6.3.3 Missed Observation

Occlusion and poor imaging conditions (e.g. dark, shadowed areas of the scene) are com-mon conditions that prevent us from observing the occurrence of some events. When we miss the recognition of one of the sub-events the whole event is missed. To prevent from this, we propose a notion of utility in the definition of the event model by associating a coefficient to each sub-event (fig. 3.9). Utility which is defined by a human expert expresses the importance or priority of sub-events for the recognition of the whole event. Its range is in the interval ]0,1], higher is the utility value higher is the importance of the sub-event in the recognition of the whole event. The value 1 means that the sub-event is required for the recognition.

3.6.3.4 Dealing with the Tracking Identifier

One of the low-level errors which deeply affects the performance of the recognition is the changes of the tracking identifier. Identity maintenance is a primary source of uncertainty for activity recognition, it affects in particular the recognition of long-term events.

¥ At the Event Modeling step:

Identity maintenance is necessary when there exist multiple identities that actually refer to the same mobile object. It is caused by lack of visual information (appearance, shape, etc.) to compute the correspondence between objects. Our approach to solve this issue at

Figure 3.10: Illustration of the definition of the ‘equal’ relation. The relation equal(p1, p2) verify whether the identifiers of the two objectsp1andp2refer to the same object.

the level of event modeling is to use specific relation in the representation of the event.

More precisely, the identification whether two objectsAandBrefer to the same object is represented by the relationequal(A, B)(see section 4.5.2, chapter 4).

The evaluation of this relation is done using appearance matching (e.g. 3D height, 3D width, etc.). Different identifying contextual cues about identities can be discussed. These cues are based on the individual belongings, closed place activity, knowledge and appear-ance as pointed in [Tran and Davis, 2008].

¥ At the Event Recognition step

The recognition of an event over time needs maintaining the same identifier for each mo-bile object when recognizing its sub-events, otherwise it will be considered as a different object. To deal with this tracking error at the event detection level, we propose the use of the recognition history of an evente,{e¹, ..., e^t−2, e^t−1}: the recognized events over time are stored in a buffer and for each time t, and for each detected evente, we propose to look at the change of its physical object identifier. If the identifier of a physical object changes suddenly and/or for a short period of time, we do not consider the new identifier and we maintain the last identifier of the physical object.

3.7 Conclusion

We have presented in this chapter an overview of the proposed approach to recognize hu-man activities. Our approach consists in combining logic and probabilistic approaches for event recognition. We have presented the different approaches proposed to manage the uncertainty of recognition during the event modeling step and the event detection step. We have also

presented the approaches proposed to manage the uncertainty of low-level data. In the next chapters, the proposed approach for activity recognition is described in details: chapter 4 de-scribes the proposed activity modeling and chapter 5 dede-scribes the approach proposed for the probabilistic event recognition. Evaluation is detailed in chapter 6.