• Aucun résultat trouvé

Social interaction in virtual reality

N/A
N/A
Protected

Academic year: 2021

Partager "Social interaction in virtual reality"

Copied!
176
0
0

Texte intégral

(1)

T

T

H

H

È

È

S

S

E

E

En vue de l'obtention du

D

D

O

O

C

C

T

T

O

O

R

R

A

A

T

T

D

D

E

E

L

L

U

U

N

N

I

I

V

V

E

E

R

R

S

S

I

I

T

T

É

É

D

D

E

E

T

T

O

O

U

U

L

L

O

O

U

U

S

S

E

E

Délivré par l'Université Toulouse III - Paul Sabatier

Discipline ou spécialité : INFORMATIQUE

JURY

Catherine PELACHAUD Professeur à IUT de Montreuil, l'Université de Paris 8,Paris. Rapporteur Daniela ROMANO Professeur à l'Université de Sheffield, Sheffield U.K. Rapporteur Jean-Pierre JESSEL Professeur à Université Paul Sabatier, Toulouse. Examinateur Bernard PAVARD Directeur de recherche CNRS-IRIT, Toulouse. Directeur de thèse

Ecole doctorale : MITT Unité de recherche : IC3-IRIT Directeur(s) de Thèse : Bernard PAVARD

Rapporteurs : Catherine PELACHAUD, Daniela ROMANO

Présentée et soutenue par PALLAMIN NICO Le 26 Septembre 2008

(2)

THÈSE

présentée devant

L’U

NIVERSITE

T

OULOUSE

III :

P

AUL

S

ABATIER

pour obtenir le grade de :

D

OCTEUR DE L

’U

NIVERSITE DE TOULOUSE

Spécialité

:

I

NFORMATIQUE Par

PALLAMIN NICO

Titre de la thèse :

Social Interaction in Virtual Reality

A soutenir le 26 septembre 2008 devant la commission d’examen

Catherine PELACHAUD Professeur à IUT de Montreuil, l'Université de Paris 8, Paris. Rapporteur

Daniela ROMANO Professeur à l'Université de Sheffield, Sheffield U.K. Rapporteur

Jean-Pierre JESSEL Professeur à Université Paul Sabatier,

Toulouse. Examinateur

Bernard PAVARD Directeur de recherche CNRS-IRIT, Toulouse. Directeur de thèse

(3)

Aknowledgment

The acknowledgment is probably the most pleasant part of a thesis to write. Not only because it is the part where you are not obliged to scientifically prove what you are saying but as well, and mostly, because when you write it you know that the thesis is over ..even if now that it is over I can say that making a thesis is a kind of fun. Stated that, there are a lot of persons to which i have to say thank you!

First to Bernard Pavard, and not only because he is my PhD Director, but also for his original way of thinking that introduced me to a different way to make research. The experience of working with you looked like a concrete application of chaos theory but for sure was never boring!

I would like to thanks the members of my reading committee, Daniela Romano and Catherine Pelachaud for sacrifying part of their summer holiday to assess my document and Jean-Pierre Jessel for accepting to be part of my jury.

A special thanks to all the people who helped me to take this adventure to an happy end, in particular my personal reviewers who corrected my english and my french (Marina, Jessica, Theodora, Elisa, Shadi, Attilio).

I would also like to thank the persons who cooperated with me in these years, Chantal, Daniela, Ludovic, Cedric, Colin, Guillaume, Mehdi, Lucila et Julie. I really thank you for your support, your help and the time spent together.

Another special thanks to Sandrine (colleague, friend and office-mate) to be always ready to help me and for the tons of « fraise tagada » and M&M’s shared when things were difficul. Candies are less effective than Prozac but much cheaper. A special thanks as well to Nadhem for all the fun, the support and to introduce me to his lovely sister ;-).

A big thank you to all the friends, in particular to Attilio, Berta, Vincenzo and Shadi. It is nice to have you around, especially when you invite me to dinner.

Another particular thanks to Laure: I will never really know if you are special or I’m masochist, but I’m still bound to you. A « gracias » as well to Matilde who prove me that after all I’m not so masochist,…so Laure you must be special, but probably not in the right way !!

Grazie of course a Mamma e Papa, always there to help me, maybe Mamma a little bit too much, but you know, « La mamma é sempre la mamma ».

And to end with a light note, I would as well thank Percy Spencer (considered the inventor of the microwave oven), to « Picard » (french frozen food reseller), « Domino-Pizza » and «La grand Pizzeria» to be the official food suppliers of this thesis. After all you can not feed a researcher only with theories !

(4)

Résumé

L’interaction sociale en univers virtuel est un domaine émergent en l’informatique et en sciences sociales qui présente des retombées potentielles très importantes dans de nombreux domaines professionnels tels que la formation, la rééducation, la conception, etc.

L’objectif de cette recherche était de qualifier ce qu’est une interaction sociale et d’identifier les conditions que doit fournir un environnement virtuel pour qu’effectivement, cette interaction sociale puisse se dérouler (se co-construire).

Nous nous sommes particulièrement intéressés à la dimension non verbale de la communication (gestuelle, posture, expressions faciales, etc.) ainsi qu’à l’articulation entre animation intentionnelle (produite par le sujet animant son avatar) et non intentionnelle (produite par un modèle informatique en fonction du contexte).

Alors qu’une grande partie des travaux dans ce domaine est basée sur un paradigme d’interaction entre humain et avatar (approche Intelligence Artificielle traditionnelle), notre approche concerne l’interaction entre des avatars animés par des humains. Notre objectif était d’identifier un ensemble de règles d’interaction susceptibles de respecter deux principes fondamentaux de l’interaction sociale : la réflexivité (le couplage dynamique entre l'acteur et son environnement) et l'indexicalité (références implicites aux objets externes de l'environnement). C’est la mise en œuvre de ces deux principes qui permettent aux acteurs humains de co-construire le sens de leur interaction en temps réel.

Après avoir développé une plateforme d’interaction en univers virtuel susceptible de reproduire ce type de phénomène, nous avons analysé la façon dont les acteurs s’approprient cet environnement virtuel et cherché à savoir si effectivement nous pouvions voir se co-construire des interactions sociales.

Dans cette optique, les acteurs humains animaient leur propre avatar dans l’univers virtuel ; ils pouvaient prendre des décisions en fonction de toutes leurs expériences passées qu’elles soient cognitives ou culturelles.

Plutôt que de représenter toutes les fonctions cognitives humaines, nous avons développé un moteur d’animation capable de reproduire la dimension dynamique de l’interaction sociale. Ce moteur permet de gérer la fusion entre l’animation intentionnelle produite par l’acteur humain (par exemple, marche avant, arrière, etc.) et les animations autonomes de l'avatar (gestes, posture du corps, expression du visage).

La validation de notre modèle est basée sur une analyse ethnométhodologique (à l’aide de vidéos et de débriefings suite aux simulations) visant à comparer les interactions sociales entre interlocuteurs en situation réel et en univers virtuel.

(5)

Abstract

Virtual reality is an emerging technology that has proved its great potentiality in various fields in the last years. Even if such technologies have been applied in different field and for various purposes, we are still just looking at the top of the iceberg. While in fact big progresses have been made concerning the realism of the visual and acoustic output and the quality of the physics simulation engines, a new branch of applications is imposing itself in a strong way. This branch concerns social interaction in virtual environments. It is just in the last years in fact that researchers start to investigate the problem of realistic and effective social interaction in virtual worlds.

The aim of our research is to improve the level of efficiency of virtual communication focusing mostly on the role of non-verbal communication. Our approach is mostly based on non-deterministic social theories that stress the role of emergence and contextual intelligence in contraposition of the classical cognitive modelling and plan-based artificial intelligence.

Considering the limits of artificial intelligence to reproduce in an effective way the complexity of human social interaction, we decide to develop an architecture able to leave the user free to exploit all his contextual intelligence to drive the social interaction. In such way we are able to grant that the characteristics of reflexivity (the dynamic coupling between the human actor and his environment) and indexicality (the implicit references to the elements of the environment) are taken in care during the interaction.

The result of our work is a virtual environment in which a set of basic social interaction rules partially drive the behaviours of the avatars. The virtual reality platform includes an animation system that handle the priority between the animations driven by the automatic modules and the one controlled by the user. The user in then always in control of his avatar and can stop the automatic animations at any moment via his voluntary control.

The validation of the model is based on a series of ethnomethodological analysis (based on video-records of the interactions and debriefing interviews) performed to directly compare similar social interactions between users in real world and in virtual environments.

(6)

Contents

1

Introduction 13

1.1 Motivations and Challenges ... 13

1.2 Objectives and Contributions ... 14

1.3 Layout of the Thesis ... 16

2

Theoretical Background Part 1: Complexity, Signs and Social

Interaction 20

2.1 Classical Social Theories ... 20

2.2 A Critic Against Reductionism: Complexity Theory ... 23

2.3 A Critic Against Symbolic Represtations: Constructivism, Constructionism and Situated Cognition ... 24

2.4 A Critic Against The Mind-Body Dichotomy: Enaction, Coupling and Social Cybernetics... 26

2.5 Ethnomethodology ... 30

2.5.1 Indexicality and Reflexivity ... 31

2.5.2 The Ethno-Methods ... 32

2.5.3 The Breaching Experiment ... 33

2.6 Semiotics ... 34

2.6.1 The Role of Context in the Interpretation of Visual Sings ... 35

2.6.2 The Role of Social Codes in the Interpretation of Visual Signs ... 36

2.6.3 Semiotics and Virtual Reality ... 37

2.7 Conclusions ... 38

3

Theoretical Background Part 2: Virtual Reality and Social

Interaction 44

3.1 Virtual Reality, Theories and Concepts: Interactivity, Immersion and Presence ... 44

3.1.1 Immersion: The Holy Grail of Virtual Reality ... 45

3.1.2 Presence ... 46

3.2 Social Interaction in Virtual Reality: The Role of Non-Verbal Behaviours ... 47

3.2.1 The Control of Actions: Direct Mapping, Indirect Mapping, Hybrid Systems49 3.2.2 The Voluntary-Involuntary Dichotomy ... 51

3.3 Meet Mr. Avatar ... 52

(7)

3.4.1 “Second-Life” and “The Sims” ... 54

3.4.2 M.R.E and Kismet ... 59

3.5 Conclusions ... 63

4

Preliminary Investigations

68

4.1 The Analysis Method ... 68

4.2 Social Interaction: Ethnomethodological Analysis ... 69

4.2.1 The Situation to be Analyzed ... 69

4.2.2 Ethnomethodological Analysis of Social Interaction ... 70

4.2.2.1 First Sample: Walking Together and Displaying Attention ... 71

4.2.2.2 Second Sample: Retrospective Pointing Gestures ... 73

4.2.3 Results and Discussion ... 74

4.3 Avatar Expressivity: Experimental Validation ... 75

4.3.1 Methodology ... 76

4.3.2 Results ... 77

4.3.2.1 The Realism of the Avatar ... 77

4.3.2.2 Comparison between the Opinions Expressed about the Avatar and about the Human ... 78

4.3.3 Conclusions and Further Developments ... 86

4.4 Conclusions ... 87

5

Design of the Virtual Reality Platform

92

5.1 Technomethodology ... 92

5.2 The Tools Used ... 94

5.3 Representational Approach Versus the Human in the Loop ... 95

5.4 The Avatar ... 97

5.4.1 The Choice of the Avatar ... 97

5.4.2 The Structure of the Avatar ... 98

5.4.3 The Animation of the Avatar ... 101

5.5 The Architecture of the Virtual Reality Platform ... 105

5.5.1 The General Structure ... 105

5.5.2 The Networking System ... 107

5.5.3 The Animation System ... 109

5.5.4 The Social Interaction System ... 112

5.6 Discussion and Conclusion ... 114

6

Social Interaction in Virtual Worlds

119

6.1 Virtual Interaction: Pilot Experiment ... 119

(8)

6.1.2 Ethnomethodological Analysis ... 121

6.1.3 Conclusion and Design Guidelines ... 123

6.2 Social Interaction Experiment ... 124

6.2.1 Experimental Setting ... 124

6.2.2 Sample of the Real World Interaction ... 126

6.2.3 Sample of the Virtual Environment Interaction ... 130

6.2.4 Ethnomethodological Analysis: Coupling Behaviour ... 135

6.2.4.1 Real Time Coupling Analysis ... 135

6.2.4.2 Interaction Style and Interpersonal Distance ... 147

6.2.5 Debriefing: Users’ Opinions ... 152

6.3 Results and Conclusions ... 153

7

Conclusions 160

7.1 Summary ... 160 7.2 Conclusions ... 161 7.3 Perspectives ... 164

Annex A

167

Reference 170

(9)

List of figures

Figure 1: Different forms of social tracking. ... 30

Figure 2: Typing animation during a chat in Second-life. ... 55

Figure 3: An avatar involved in a voice chat in Second-life. ... 55

Figure 4: An avatar performing a waving-hand animation in Second-life. ... 56

Figure 5: A death sequence in the Sims: on the left the death appear, in the middle the Sim trying to reverse the sand clock to have more lifetimes, on the right the relatives crying the death of the Sim. ... 57

Figure 6: Sample of social interaction on the Sims. ... 58

Figure 7: The M.R.E. virtual environment. ... 60

Figure 8: Samples of Kismet face expressions... 61

Figure 9: Kismet reactions to optimize interaction. ... 62

Figure 10: Left: Fireman on the lower balcony proceeds to climb the ladder to reach the victims (one of which is being suspended over the balcony) on the second floor. Right: The second victim awaits rescue after helping the evacuation of the first victim. Notice the smoke at the window. ... 70

Figure 11: SC (on the left) talking to the GC wearing an orange suite. ... 72

Figure 12: SC looking at the GC while talking. ... 72

Figure 13: Dialogue and gestures between the GC and the SC. ... 73

Figure 14: Summary of the most common remarks concerning the realism of the avatar. ... 77

Figure 15: Summary of the most common remarks in favour of the avatar’s realism. ... 78

Figure 16: Virtual agents and human pointing toward a referent. ... 79

Figure 17: Virtual agent and human performing an open-hands gesture. ... 80

Figure 18: Virtual agent and human expressing temporal succession. ... 82

Figure 19: Virtual agent and human performing a self-contact gesture. ... 83

Figure 20: Creation of a textured 3D head using a side and a front picture. ... 86

Figure 21: A representational intelligence view shown on the left; on the right is the contextual intelligence view (Dugdale 1999). ... 96

Figure 22: A model of our avatar showing the skeletal structure. ... 98

Figure 23: Sample of U.V. map of the avatar clothes. ... 99

Figure 24: Sample of U.V. map of the avatar face. ... 99

Figure 25: Graphical representation of the avatar’s facial bones. ... 100

Figure 26: Schematic representation of a chain of bones used in inverse kinematics. ... 101

Figure 27: Example of animation graph used in videogame industry. ... 102

Figure 28: Retargeting of animation using “Life Form Studio 4”. ... 104

Figure 29: Schematic representation of the virtual reality platform’s architecture. ... 106

Figure 30: Structure of the “Manage Distributed Classes” building block that controls the similarity of the simulation in the different computers. ... 107

Figure 31: Sample script that handles the process of broadcasting messages trought the local network. ... 108

Figure 32: Sample image of the “unlimited controller” building block. ... 110

Figure 33: Example of script used to control voluntarly the head rotation of the avatar. ... 111

Figure 34: A sample of the implementation of interaction rules in the virtual world. ... 112

Figure 35: Module in charge of the detection of others avatars in the field of view. ... 113

Figure 36: Coordination of activities through indexical gestures in the real world. ... 121

Figure 37: Left: Narrowing the field of view in the virtual world (the subject is unable to see both the pointing arm and the pointed object at the same time). Right: the user pointing at the screen instead of exploiting the pointing capabilities of her avatar.122 Figure 38: The “Third person view” adopted in the first virtual session. ... 125

Figure 39: The “First person view” adopted in the second virtual session. ... 125

(10)

Figure 41: Identification of gaze direction in real world. ... 136

Figure 42: Gaze direction performed in response of a deictic in the real world. ... 136

Figure 43: Subject staring at an object in the virtual world (subjective view). ... 137

Figure 44: Participant performing a pointing gesture in the virtual world (third person view).138 Figure 45: Change in the field of view as reaction to a pointing (subjective view). ... 138

Figure 46: Delay on performing a pointing gesture (semi-transparent view). ... 139

Figure 47: Subject virtually touching the book to confirm his understanding (third person view). ... 139

Figure 48: Subject performing a pointing gesture to confirm his understanding in the virtual world (semi-transparent view). ... 140

Figure 49: Automatic head orientation toward the speaker performed in virtual world (third person view). ... 141

Figure 50: Voluntary rotation of the head to look at the speaker (semi-transparent view). ... 142

Figure 51: Automatic head rotation toward a potential addressee of the conversation (third person view). ... 143

Figure 52: Automatic head orientation toward an avatar that is not the addressee of the message (third person view). ... 143

Figure 53: Voluntary head orientation toward the addressee of a message (subjective view). 144 Figure 54: Voluntary head orientation in the direction suggested by the other avatar gaze (semi-transparent view). ... 145

Figure 55: Mirroring of hand gesture during real world interaction. ... 146

Figure 56: Synchronous pointing gesture in the virtual world. ... 146

Figure 57: Mirroring of hand gesture while performing a virtual world interaction. ... 147

Figure 58: Graph comparing the time needed to complete the task in the four settings. ... 148

Figure 59: Subject trying to pass behind an avatar that is obstructing his way (third person view). ... 149

Figure 60: Physical contact during a real life interaction. ... 150

Figure 61: Comparison between interpersonal distance in the real and in the virtual world performed by the same subjects. ... 151

Figure 62: Graphical representation of “personal space”. ... 155

Figure 63: The three levels of interaction in our system. ... 163

Figure 64: Design of the control room. Bottom left: enlargement of the virtual terminals space with two operators speaking altogether. ... 165

(11)

List of tables

Table 1: Average judgment of the participants for the first sequence.

*Difference between the two groups statistically significant (p<0.05). ... 79

Table 2: Average judgment of the participants for the second sequence.

*Difference between the two groups statistically significant (p<0.05). ... 80

Table 3: Average judgment of the participants for the third sequence.

*Difference between the two groups statistically significant (p<0.05). ... 82

Table 4: Average judgment of the participants for the fourth sequence.

*Difference between the two groups statistically significant (p<0.05). ... 84

Table 5: Samples taken from the real world interaction. ... 129 Table 6: Samples of social interaction performed in the virtual world adopting a third person

(12)

Résumé

CHAPITRE 1

La réalité virtuelle est devenue de plus en plus populaire ces dernières années et a attiré l'intérêt de plusieurs instituts de recherche. La conséquence d'un tel intérêt est que les applications de réalité virtuelle touchent autant l'industrie du divertissement, que les domaines de la visualisation de données, la formation et plus récemment, la psychothérapie. Un des challenges de ces nouveaux champs de recherche concerne les interactions sociales dans les environnements virtuels.

Même si de plus en plus de chercheurs sont impliqués dans le domaine de l'interaction virtuelle, les interactions sociales en monde virtuel sont toujours loin d'atteindre le niveau de naturalisme et d’efficacité rencontrés dans le monde réel. Ce manque de solutions efficaces est attribuable à différentes causes. La première est naturellement la jeunesse relative de ce champ de recherche. Une autre cause est due au fait que la recherche sur l'interaction virtuelle implique une approche interdisciplinaire difficile à mettre en oeuvre. Une telle recherche implique des disciplines historiquement et méthodologiquement très différentes, comme par exemple avec les sciences sociales et l'informatique. La dernière cause, et probablement la principale, concerne le manque d'une connaissance profonde de la dynamique régissant les interactions sociales.

Le travail présenté dans cette thèse montre notre tentative d'augmenter l'efficacité de l'interaction sociale en monde virtuel en se référant à quelques théories sociales innovatrices basées sur le non-déterminisme.

Ces nouvelles théories sociales, comme celles de l'ethnomethologie et de l'enaction se sont concentrées sur la complexité de l'interaction sociale et sur son caractère imprévisible. À partir de ces trames théoriques, nous décidons d'éviter une approche classique comme celle adoptée en intelligence artificielle. L'inefficacité de modelage cognitif classique nous a guidé vers un paradigme qui intègre les utilisateurs dans le système. Plutôt que d’essayer de substituer le savoir faire humain par l'intelligence artificielle, nous avons développé une architecture qui donne à l'utilisateur la possibilité de contrôler la situation au maximum.

En nous basant sur les travaux de Brooks au sujet de l'architecture de « subsumption », nous avons développé un système hybride pour la commande des avatars. Dans notre système, un ensemble de règles d’interactions sociales est intègré dans la plateforme de réalité virtuelle.

Le résultat final de notre travail est un environnement virtuel coopératif. L'architecture développée permet à plusieurs utilisateurs d’interagir dans le même environnement virtuel. L'architecture finale inclut trois modules responsables de différentes tâches. Le premier module est en charge de la distribution de la simulation par le réseau local. Le deuxième module concerne le système d'animation.

(13)

Le troisième module est responsable des interactions sociales automatiques. Cet environnement virtuel coopératif est évalué à partir d’une série d'expériences ethnomethodologique.

La thèse est structurée en sept chapitres.

Le chapitre 1 présente le sujet de la thèse, nos contributions pour d'augmenter l'efficacité de l'interaction sociale dans le monde virtuel et présente l’exposition du plan adopté pour structuré le document.

Le chapitre 2 décrit les théories sociales des sciences constituant le fond théorique de notre travail. Il débute par une introduction aux théories sociales classiques en présentant deux disciplines ; l’ethnomethodologie et la sémiotique, qui ont fortement influencé notre recherche.

Le chapitre 3 présente des concepts et des théories dans le champ de la réalité virtuelle. Après présentation de certains concepts généraux comme l'immersion et la présence, ce chapitre se focalise sur l'interaction sociale en monde virtuel. La dernière partie du chapitre décrit quelques applications consacrées à l'interaction virtuelle.

Le chapitre 4 expose l'analyse exécutée pour identifier les éléments sociaux d'interaction que nous devons mettre en application dans notre plateforme pour supporter une interaction sociale pertinente. Cette analyse est divisée en deux sections. La première concerne l'analyse ethonomethodologique réalisée afin d'identifier les règles d'interaction qui rendent l’interaction sociale pertinente. La deuxième section concerne l'expérience entreprise pour valider les capacités de communication non-verbales de nos avatars.

Le chapitre 5 traite, quant à lui, du développement technique de notre plateforme. Le chapitre présente l'approche technomethodologique que nous avons adoptée et conclue en présentant les éléments que nous avons mis en application dans notre environnement virtuel.

Le chapitre 6 présente les résultats des expériences conduites avec notre plateforme. Le chapitre présente dans un premier temps l'expérience pilote que nous avons exécutée pour définir au mieux les cadres expérimentaux à appliquer dans la simulation principale. Le reste du chapitre décrit l'expérience principale et ses résultats. L'analyse concentrée sur divers éléments est basée sur différentes techniques d'analyse. La présence du comportement de couplage par exemple a été analysée avec un protocole ethonomethodologique, tandis que les différences dans le style d'interaction étaient basées sur des méthodes d'observation classiques.

La conclusion générale est présentée en chapitre 7. Après la présentation de la synthèse des principaux résultats et apports de notre travail, nous en discutons les apports pour la conception d’environnements virtuels réalistes d’un plan social et présentons quelques perspectives de recherche.

(14)

Chapter

1 Introduction

1.1 Motivations and Challenges

Virtual reality has been more and more popular in the last years. The popularity of this technology is bound to two factors. The first one concerns the increasing computational power of modern hardware that makes virtual reality available to a big part of the customers. The second factor, that is partially the consequence of the first, refers to the fact that research institutions and private companies are foreseeing the potentiality of virtual reality technologies and investing in it. The consequence of such investments is that the range of virtual reality applications is spreading from entertainment industry to data visualization, training and, in these last years, psychotherapy.

The quality of synthetic graphics and localized audio has reached levels of sophistication so high that they can produce audio and video experiences able to fake our senses. This is particularly true when the applications are free from the constraints of real time. In such cases, the quality of synthetic images is so accurate that they become indistinguishable from real video images, making them the perfect candidate to furnish special effects in movies.

This increased realism induced that, part of the virtual reality’s research community has shifted its attention to new fields of investigations. One of the most challenging and prolific of these new research fields concerns social interactions in virtual environments.

The research of social interaction in virtual worlds is not confined to the academic court. The popularization of internet has permitted the emergence of shared virtual spaces where different users can interact one with each other through a 3D virtual representations of themselves (so colled avatar).

Some of these virtual spaces can reach several millions of users and their ability to support social interaction is their main appeal (Second-Life counts almost 8.000.000 users and I hope to finish my thesis before it reaches 10.000.000). Information technologies experts see social interaction as the seventh wave of virtual reality. Even if there is a consensus about the huge potentiality of virtual reality for social interaction, these potentialities are still clearly underexploited.

The lack of efficient solutions for social interaction in virtual worlds is ascribable to different causes. The first one is of course the relative youth of this research field. Another cause is that the research on virtual interaction implies a hard interdisciplinary approach. Such research involves fields historically and

(15)

methodologically very different, like social science and computer science. The last cause and probably the main one, is the lack of a deep knowledge of the dynamics governing social interactions.

Social sciences in fact are still not able to present a general model of social interactions. Recent models are based on an epistemological background that denies determinism in favour of an opportunistic and emergent vision of social interaction. These new branches of social sciences (like enaction theory and ethnomethodology) seam quite promising but, due to their refuse of a deterministic approach, they are complex to integrate into a computational model. Informatics is indeed still historically bound to procedural and classical AI methods that are developed in function of a predictable world (Pavard 2002).

The work presented in this thesis represents our attempt to integrate these new social paradigms, based on non-determinism and emergence, in the design of a virtual reality platform. In particular, our aim is to increase the efficiency of social interaction in virtual environment focusing on the role played by indexicality and reflexivity. These two concepts, which we will introduce in chapter 2 while talking about ethnomethodology, are at the base of a natural social interactions.

1.2 Objectives and Contributions

The aim of our work is to investigate social interaction in virtual environments. In particular, we focus on the role played by non-verbal communication and the way in which the non-verbal communication can be implemented and exploited in virtual worlds.

Our prime interest lies in the unconscious gestures, normally performed during social interaction. Such gestures, without conveying an explicit message, play a main role in the correct interpretation of the information. During interaction, these subtle non-verbal communications provides keys to decode the interpretation frame of the interaction. At the same time these gestures, in particular gazes, play a crucial role as back-channels to structure the course of the conversation.

Gestures, postures and face expressions are at the basis of the social coupling phenomena. The possibility to interpret and exploit social coupling is a fundamental prerequisite for natural social interaction in virtual environments.

Our work is strongly influenced by the new wave of social sciences that rejects the cognitive vision of humans as rational problem solvers. These new theories, like ethnomethology and enaction theory, have focused on the complexity of social interaction and its unpredictability. Starting from these theoretical frames, we decide to avoid a top down approach, like the one adopted in classical artificial intelligence. The inefficiency of classical cognitive modelling drove us to adopt a “human in the loop” paradigm that integrates the users in the system. Instead of trying to replace humans’ skills with artificial intelligence, we developed an architecture that gives the user the possibility to exert the maximum control on the situation.

Based on the works of Brooks (Brooks 1991) concerning subsumption architecture, we developed an hybrid system for the control of the avatars. In our

(16)

system a set of basic social interaction scripts are implemented in the virtual reality platform. These scripts are integrating (and interacting with) the actions executed by the user driving his avatar.

The choice to implement a virtual environment that integrates user’s voluntary action with automatically generated social reactions, implies a twofold research query.

The first question concerns social sciences, and focus on the identification of the set of automatic actions (interaction’s scripts) that is suitable to increase the effectiveness of social interactions in virtual environment.

The second set of question is connected with cybernetics and regards the three different levels of interactions that we could observe in the case of virtual communication. The first level concerns how the two systems (the human and the avatar) are interacting. In particular, we are interested in understanding how the user will incorporate and exploit the gestures automatically generated by the system. The second level deals with the interactions between the avatars. In this case, we are interested in investigating the presence, in the avatar-avatar interaction, of coupling level similar to the ones that exist in the human-human interaction. The third level concerns the human-human interactions mediated by the virtual reality. In particular, we are interested in investigating how the automatic gestures will affect inter-human social interaction.

To answer these issues we developed a serie of methodological and practical tools. The methodological contribution is represented by the design and evaluation protocols that we applied in our researches. The practical contribution is represented by the virtual reality platform that we developed to run our simulations.

The methodology that we applied to the design and evaluation of our virtual reality system is mostly based on ethnomethodological analysis. Ethnomethodology is usually applied to study social interactions. In our study, we decided to apply this method to identify, through analysis of real social interaction videos, the elements that need to be incorporated in our virtual reality platform in order to increase the efficiency of social interaction. The originality of our work consisted in applying the etnomethodological analysis, not only in the design phase, but also for the evaluation of virtual interactions. In this second case, the analysis did not concern interaction between real persons but between avatars. As we will see in chapter 6 in fact, the evaluation of our platform is based on the comparison of social interaction performed in real life with interactions performed by the avatars in the virtual world.

The practical contribution consisted in the development of a cooperative virtual environment. To achieve this goal we developed an architecture that allows multiple users to inhabit the same virtual environment. This architecture includes three modules in charge of different tasks. The first module is constituted by a set of scripts dedicated to the distribution of the simulation through the local network. The second module concerns the animation system. The third module is in charge of the automatic social interactions.

A last important contribution of our research work concerns the design, creation and animation of the virtual characters. Considering the important role that avatars play in virtual interaction, we dedicated a particular attention to the development of

(17)

our characters. The design of our avatars is grounded on a methodology that starting with a serie of preliminary video analysis of human interactions and includes an experimental validation to evaluate the non-verbal communications skills of our characters. The structure proposed for the virtual character’s body is based on a virtual skeleton that mimics the human morphology and that grants the high degree of freedom needed to create realistic and efficient animations.

1.3 Layout of the Thesis

The document is structured in seven chapters.

Chapter 1 introduces the topic of the thesis, presents our contributions in order to increase the efficiency of social interaction in virtual world and finishes with the layout of the thesis.

Chapter 2 describes the social sciences theories constituting the theoretical background of our work. The chapter begins with an introduction to classical social theories. The limits of these theories, based on reductionism and representationalism, are then analyzed in contraposition with some alternative views, like constructivism and enaction. The chapter ends introducing two disciplines, ethnomethodology and semiotics that have strongly influenced our researches.

Chapter 3 presents concepts and theories from the field of virtual reality. After introducing some general concepts like immersion and presence, the chapter focuses on social interaction in virtual worlds. In particular, we present some studies investigating the role of non-verbal communication and avatars in virtual interaction. The last part of the chapter depicts some relevant applications dedicated to virtual interaction.

Chapter 4 describes the analysis performed to identify the social interactions elements that we need to implement in our platform to support effective social interaction. This study is split in two sections. The first one concerns the ethonomethodological analysis performed in order to identify the interaction rules that make social communication effective. The second part concerns the experiment conducted to validate the non-verbal conversation skills of our avatars.

Chapter 5 discusses the technical development of our virtual reality platform. The chapter starts presenting the technomethodology approach that we adopted in the design phase. Our design focuses on minimizing the use of Artificial Intelligence in favour of a high degree of control by the user. The chapter ends presenting the elements that we implemented in our virtual environment with the aim to support effective social interaction. The design of the platform is strongly based on the results of the preliminary studies presented in chapter 4.

Chapter 6 presents the results of the experiments conducted with our virtual reality platform. The chapter starts presenting the pilot experiment that we run to better define the experimental settings to be applied in the main simulation. The rest of the chapter describes the main experiment and its results. The experiment consisted in a serie of social interactions that are performed, firstly in the real world and then in the virtual environment. The analysis focused on various elements and is

(18)

based on different techniques of analysis. The presence of coupling behaviour for instance was analysed with an ethonomethodological protocol while the differences in the interaction style were based on classical observational methods.

The document ends with chapter 7 that, after summarizing the results of our work, draws some conclusions and presents some future perspectives.

(19)

Résumé

CHAPITRE 2

La psychologie cognitive est probablement une des doctrines qui a le plus influencé les sciences sociales ces dernières années. Elle définit la fonction de la cognition par l’application d’algorithmes (ensembles de règles) qui servent à manipuler notre représentation du monde. Cette perspective a fortement influencé les recherches dans les sciences sociales et a donné naissance à des théories comme la "Theory Of Mind". La "Theory Of Mind" est essentiellement focalisée sur la façon dont, au cours d’une interaction sociale, les gens se comprennent.

La psychologie cognitive et la "Theory Of Mind " représentent deux échantillons des théories sociales classiques. Les théories sociales classiques sont caractérisées par trois éléments principaux :

• Le premier concerne l'adoption d’une aproche réductionniste.

• Le deuxième élément réfère à l'introduction de la dichotomie entre le corps et

l’esprit rendue populaire par la distinction entre "Res Cogitans" et "Res Extensa" théorisée par Descartes.

• Le troisième élément, qui est en partie une conséquence du précédent relève

de l'adoption d'un paradigme de représentations symboliques comme base des capacités cognitives humaines.

Ces trois éléments ont été durement remis en question ces dernières années suite à l’apparition de nouveaux paradigmes scientifiques.

Les disciplines comme la théorie de complexité et la cybernétique ont de forts arguments contre l'utilisation du réductionnisme dans les sciences.Le concept réductionniste (base de l’expérimentation) déclare que nous pouvons comprendre des phénomènes naturels complexes en les ramenant à l'interaction de leurs éléments, ou en d'autres termes, que le système n’est que la somme de ces éléments. Il s'est avéré qu'une telle vision, tout en étant appropriée pour expliquer quelques phénomènes physiques, est inadaptée, à l’analyse des systèmes complexes. Les systèmes complexes, comme les systèmes sociaux par exemple, sont souvent caractérisés par des phénomènes de niveau élevé qui ne sont pas prévisibles par l’application de règles basiques.

Le deuxième élément qui caractérise des théories sociales classiques est la dichotomie Cette dichotomie « corps-esprit » (deuxième élément évoqué précédemment) a été critiqué par la théorie de l'enaction. Cette théorie expose que la connaissance est construite sur l'habileté motrice et considère que l’esprit est inséparable de sa structure organique.

(20)

Le paradigme symbolique de représentation (dernier élément évoqué), a été critiqué par différentes théories sociales comme le constructivisme, le constructionisme et la cognition située. Ces théories rejettent la vision de l'humain comme solutionneur de problème en appliquant des règles abstraites. A l’opposé, elles soulignent le rôle du contexte et de la situation réelle dans le processus décisionnel.

Ces dernières années, en réaction à cette critique des nouvelles théories ont adopté une perspective différente dans l'étude de l'interaction sociale. Parmi ces théories deux sont particulièrement appropriés pour nos études : ethnométhodologie et sémiotique.

L’ethnométhodologie est une branche des sciences sociales créée par Garfienkel. Son objet d'étude est la façon dont les gens comprennent l'environnement social qui les entoure. L’ethnométhodologie, au lieu d'analyser les causes d'une transmission, est intéressé par le processus continu de transmission. Selon l'ethnométhodologie, toutes les significations dépendent d’un contexte et par conséquent sont subjectives.

Dans la trame de notre recherche l'ethnométhodologie a été utilisée pour l'identification des règles d’interactions sociales. Ces règles ont ensuite été intégrées dans notre modèle et dans l'évaluation finale de notre plateforme virtuelle présentée en chapitre 6.

La sémiotique, quant à elle, peut être vue comme une champ d'étude complémentaire de l'ethnométhodologie. Alors que que l'ethnométhodologie se concentre sur le processus interne de communication, la sémiotique la plupart du temps se concentre sur la signification des signes transmis. La sémiotique étudie le rôle des signes et des systèmes de signes dans la création et la transmission. Dans cette perspective, la signification d'un signe n'est pas statique, mais change dynamiquement en fonction de notre vision du contexte. La traduction du signe dépend du contexte physique dans lequel a lieu la transmission et dépend de l’ensemble des codes sociaux. L'intégration des codes sociaux dans la traduction sémiotique provoque la distinction entre la signification connotative et la valeur d'un signe (signification « denotational »). Les significations connotatives sont celles développées par l'environnement social (traductions sociales) et habituellement elles ne représentent pas les qualités initialement signifiées par la signification « denotational ». La présence d'une signification connotative est primordiale dans l'interaction sociale et réfère à la complexité et l'ambiguïté du système de codage. Cette incertitude de la signification connotative explique le rôle important du contexte pour enlever toute ambiguïté. Dans notre recherche, la sémiotique nous a guidé dans la première partie de nos expériences sur l'expressivité et les gestes de nos avatars, comme nous le décrivons dans le chapitre 4.

(21)

Chapter

2 Theoretical Background Part 1:

Complexity, Signs and Social Interaction

The aim of our investigation is to unveil how social interaction unfolds in virtual worlds and how such interactions differ from the ones we found in real life.

In this chapter we will describe the theoretical backgrounds that have driven our explorations. In particular we will present a set of theoretical frames characterized by a similar vision of social interaction based on the reject of deterministic (rule based) models in favour of ever-changing dynamic systems characterized by indeterminism and emergent phenomena.

This chapter will start with a short presentation of a classical social approach based on the scientific determinism and strong mind-body dissociation.

After such presentation we will present some of the critics to this vision of social interaction introducing some alternative based on a constructivist approach.

The chapter will then finish with a detailed presentation of two fields of research (ethnomethodology and semiotics) that have strongly influenced our methodology and consequently the design of our virtual reality platform.

2.1 Classical Social Theories

The study of social interaction has rarely been considered as an independent field of studies but has often been a part of various fields of research ranging from psychology to anthropology and sociology. These studies have been influenced by three main elements.

The first one is the adoption of the scientific method that promotes the use of experiments to validate theories. An experiment is a study conducted to identify the link of causality between two variables. In a typical experimental setting, the experimenter observes the variation of a phenomenon in function of his manipulation of one variable while keeping the others elements constant. The advantages of the experimental methods are the identification of the causality relation between variables, the possibility to reproduce the results and the high level of objectivity obtainable.

The second element is the introduction of the dichotomy between body and mind made popular by the distinction between “Res Cogitans” and “Res Extensa” proposed by Descartes.

(22)

The third element, which is in part a consequence of the mind-body dichotomy, is the adoption of a symbolic representational paradigm as a frame for the study of human faculties.

Cognitive psychology, one of the most influencing schools in this last years, is probably the clearest example of such influences. Cognitive psychology stress the role of cognition in human life and see humans’ problem solving capability as the result of the application of algorithms (sets of rules) to manipulate the user’s internal representation of the world. The parallelism between cognitivism and informatics is so evident that often computers have been used by cognitive psychology as metaphors for the human mind. In accordance with the Descartes’ mind-body dichotomy cognitivists see the thinking process, (the mind) independent by the material element (the brain) that represent just a physiological substratum for the process, in the computer analogy the mind and his function are seen as the software while the brain represent the hardware.

Even if cognitive psychology mostly focused on the study of high level mental processes like problem solving and logical reasoning, this vision has strongly influenced some social psychology theories like “Theory Of Mind”.

“Theory Of Mind” (Baron-Cohen 1995; Davies 1995); has mostly focused on how, during a social encounters, people understand one each other. The focus of this psychology paradigm is on how we can interpret and predict each other’s behaviour. “Theory Of Mind” represents a classical social theory based on a deterministic cognitive model. Such theory sees the existence of mental process as a symbolic manipulation, and assumes that it is possible to draw inference about the mental state of the interlocutor. Such inference according to the theory is based on algorithms that guide the interpretation of the other person’s behaviour. “Theory Of Mind” focus on the cognitive skills of the subject and his symbolic representation, while the interaction process is seen as the mere product of the actors actions.

Other theories that have focused on social interaction have, at least in part, abandoned this anthropocentric vision. “Speech Act Theory” for instance is mostly involved in the study of the utterances performed in communicative acts, while “Conversation Analysis” sees in the mechanisms and sequential structure of conversation the element on which social order is founded. This last approach was mostly developed by Harvey Sacks in the 1960’s. “Conversation Analysis” was at the beginning a methodology used in sociology and only later has been adopted by linguistic and anthropology. “Conversational Analysis” in contrast with classical linguistic, states that the meanings are not taken by the participants, but that the meaning is actually created and negotiated during the interaction. Sacks, in his work, abandons the experimental approach based on reductionism and hypothesis testing. With the aim to unveil the underling structure of social interactions, his methodology is based on the observation of spontaneous conversation in natural settings. Sacks’ work mostly focused on how conversations are conducted in real world and in particular, how the interlocutors exploit turn-taking and topic management to structure the information exchange (Sacks 1974).

While conversation analysis focuses mostly on the role of verbal communication, “Context Analysis” broadens his field of interest to the non-verbal communication (Kendon 1990). “Context Analysis” found his background in interpersonal

(23)

psychiatry and the interactionist social science that define the self, not as an a priori entity, but like the result of the social interaction with the others. Interaction is therefore seen as central in understanding social behaviour and consequently it becomes a topic of investigation itself. Like in “Conversation Analysis”, the methodology is based on observation and annotation, and avoids the experimental method. Even if the work of Kendon stressed the role of interaction, his vision is still individual-based. The interaction in fact is never seen as an autonomous process but is analyzed in relation with the information that it can transmit about the actors.

These last years, the adoption of the experimental method for the study of human behaviour, even if largely diffused, has been strongly criticized. “Conversation Analysis” and “Context Analysis” that we have just introduced, represent two examples of theories that, at least in part, chose to abandon the scientific method.

The critics of the scientific method in social research are, on one hand, based on pragmatic reasons and, on the other hand, dictated by a philosophical shift in science.

Among the practical critics, we can account the high degree of artificiality induced in the experimental settings that limits the ecological validity of the results (the transposition of the experimental results on real life settings). Another critic concerns the need to control the interfering variables in the observed phenomenon that is a “condition sine qua non” for an experiment to be scientific. Such precondition is hardly obtainable when the experiment concerns humans, in fact, while the experimenter can control the external stimuli, he has no control on the internal states (emotions, mood, motivations) that could play a main role in the execution of the experimental task.

The philosophical criticism on the other hand concerns two concepts that are at the base of this method: determinism and reductionism.

Determinism is the main philosophical base that supports the principle of hypothesis validation trough experiments. Determinism states that every event, including social interaction and human cognition, is the consequence of a chain of prior occurrences (Van Inwagen 1983). This implies, in his radical view, a scientific positivism in which all natural phenomena can be predicted once we have the complete knowledge of all natural laws.

Reductionism, which is partly connected with determinism, states that we can understand natural complex phenomena by reducing them to the interaction of their elements, or in other words, that the system is just the sum of his elements. Such vision, while being appropriate to explain some physical phenomena, was proven to be inadequate when applied to more complex systems. Complex systems (like social systems) in fact are often characterized by high-level phenomena that are not predictable applying those natural laws. Disciplines like cybernetics have strong arguments against the use of reductionism in science, proposing different views like in second-order cybernetics (Heylighen 2001).

As we will see further in this chapter, reductionism has been strongly criticized not only when applied to social science, but in the last years its validity has been questioned by the emergence of complexity theory even in “hard-sciences” like physics.

(24)

2.2 A Critic Against Reductionism: Complexity Theory

Complexity theory, with its focus on interactions, has opened a new way to analyse social system and gave a new key of interpretation to collective phenomena like cooperation and sharing of knowledge. This new way to interpret phenomena has strongly influenced social science in the recent years, contributing to change the vision of human from an information processing unit to a more complex and dynamic entity, sensible to the context he lives in.

A complex system is a system for which it is difficult, if not impossible, to restrict its description to a limited number of parameters (or characterizing variables), without losing its essential functional properties. The theories of complex systems have been developed along three complementary, but nevertheless distinct, axes: the theory of non-linear systems, the neural network approach and the theory of distributed self-organizing systems.

Historically, the notion of complex systems was born at the beginning of the century when H. Poincaré was studying equations to predict the trajectories of planets. H. Poincaré showed that it was mathematically impossible to find an exact solution to these equations, even for a system as simple as those containing three planets, due to the fact that they are interacting in a non-linear way. The researches of Poincaré (Poincaré 1892-99) pointed out that the complete knowledge of the rules of a system is not sufficient to determinate in an exact way its future behaviour, proving that even very simple systems can give rise to complex and unpredictable behaviours.

The adoption of complexity theory in social science is connected with the study of distributed and self-organizing systems that have provided new perspectives in modelling social and cognitive structures. Basically, the theory of distributed and self-organizing systems states that a population of independent and autonomous agents interacting only locally may produce “intelligent” global behaviour.

A system starts to have complex behaviours (usually non-predictability and emergence) when its parts interact in a non-linear way. A really complex system would be completely irreducible. This means that it would be impossible to derive a model of the system without losing its properties.

A complex system is usually characterized by the following properties (Pavard 2000):

1) Non-determinism and non-tractability. A complex system is fundamentally non-deterministic. It is impossible to anticipate precisely the behaviour of such systems even if we completely know the function of its constituents.

2) Limited functional decomposability. A complex system has a dynamic structure. It is therefore difficult, if not impossible, to study its properties by decomposing it into functionally stable parts. Its permanent interaction with its environment and its properties of self-organization allow it to functionally restructure itself.

3) Distributed nature of information and representation. A complex system possesses properties comparable to distributed systems (i.e. some of its functions

(25)

cannot be precisely localized). In addition, the relationships that exist within the elements of a complex system are short-range, non-linear and contain feedback loops (both positive and negative).

4) Emergence and self-organization. A complex system comprises emergent properties that are not directly accessible from an understanding of its components.

This last property is of particular relevance for the social science. Emergence is the process of deriving some new and coherent structures, patterns and properties in a complex system. Emergent phenomena occur due to the pattern of interactions (non-linear and distributed) between the elements of the system over time. One of the main points about emergent phenomena is that they are observable at a macro-level, even though they are generated by micro-level elements. All this characteristics are strictly connected with social phenomena that are in the most of the cases the result of individual behaviours (I.e. the presence or social norms within a group).

Particularly relevant in the study of human interaction is then the distinction made by Gilbert (Gilbert 1995) between first and second order emergence. The differentiation between human social organizations compared to non-human social organizations (such as a collection of ants) lies in the ability to reason. Specifically, “people have the ability to recognize reason about and react to human institutions, that is, to emergent features. Behaviour which takes into account such emergent features might be called second-order emergence (as opposed to first-order emergent behaviour)”. This takes strong implications when analysing human’s social situations since we might need to consider what consequences these macro-level properties have on individuals.

2.3 A Critic Against Symbolic Represtations: Constructivism,

Constructionism and Situated Cognition

Different theories have investigated the phenomenon of human interactions under various points of view. In this section we will introduce those fields of social science (constructivism, constructionism, situated cognition) that have taken a distance from the cognitive vision of humans as a manipulator of mental symbolic representations.

The theories that we will introduce in this section are, at least in part, the consequence of the phenomenological revolution proposed by Edmund Husserl. Phenomenology is a branch of philosophy that criticizes the conventional science stating that is too abstract and too far from our personal experience. Husserl envisions, in opposition to this conventional science, a discipline that was based on our personal (phenomenological) experience. Phenomenology, with its focus on everyday personal experience tries to reject those formalized reasoning that tent to identify abstract and general rules, while it tries to discover the connection between the representation that we create of an object in our mind and the actual perceptions that we have on it. A further development of this concept can be found in Heidegger when he states that we interpret the meaning through the way we encounter the world, pointing out the central role of our engaged participation in the world for the creation of our representations.

(26)

As we will see further this Husserlian re-evaluation of subjective experience has been highly incorporated in others fields of social science like ethnomethodological studies (presented in the next paragraphs) that strongly rely on the subjective experience of the participant-observer as a main tool of analysis.

In accordance to these visions, social constructivism focuses its attention on the way the knowledge is generated. According to these branch of social science, knowledge is not passed from one person to another (for instance from the teacher to his student) but is built by the apprentice. In the epistemological view of constructivism, every knowledge is “constructed” and it does not reflect any “transcendent” external reality but it is the result of human perceptions, social conventions and personal experiences.

In accordance to the assertions of Di Vesta (Di Vesta 1987) that state “there is not a set of generalized learning laws with each law applying to all domains", constructivism paradigm stresses the central role of the context in the process of learning (McMahon 1997). Decontextualized knowledge is then rejected considering that it could be ineffective in those task where the learned concepts are interacting with a complex situation. This is particularly true when is the interaction between the knowledge acquired and the context that is relevant (Duffy 1992). Context, in the constructivism assertion, is not only the physical environment on which the learning process take place but it includes even the social context that the learner bring with him (Grendler 1997).

Constructivist learning designers have identified a number of key characteristics that enable meaningful learning to take place (Jonassen 2003). The following list of characteristics stresses how virtual reality technology is a particularly adequate and involving learning tool.

• Active - Learners are engaged by the learning process in mindful processing of information where they are responsible for the result.

• Constructive – Learners integrate new ideas with prior knowledge in order to make sense or reconcile a discrepancy, curiosity, or puzzlement.

• Collaborative - Learners naturally work in learning and knowledge building communities, exploiting each other's skills while providing social support, and modelling and observing the contributions of each member.

• Intentional - All human behaviour is goal directed.

• Complex - Problems are multiple components with multiple perspectives and cannot be solved in predictable ways

• Contextual - Learning tasks that are situated in some meaningful real world task are not only better understood, but also more consistently transferred to new situations.

• Conversational - Learning is inherently a social, dialogical process.

• Reflective - Learners should be required by technology-based learning to articulate what they are doing, the decisions they make, the strategies they use, and the answers that they find.

(27)

A similar approach to the one exposed by constructivism is adopted by constructionism. One of the differences between these two theories is that constructionism focuses more on the collective and cooperative role of knowledge creation. According to constructionism the learning process is more effective when the apprentice is involved in the creation of a meaning that is socially shared. Constructionists are particularly interested in the identification of the ways in which the singles and the groups interact in the creation of their representations of the world. This representation of the world is the result of an ongoing dynamic process driven by the way people act. The representation of a situation is then influencing the actions of the different persons, and recursively such actions (and their interpretation) contribute to the evolution (that often goes in the direction of a validation or reinforcement) of such representation. In this case like stated by Ernst von Glaserfeld (Glasersfeld 1995) all knowledge is created by a self-regulatory process of the mind, and as a consequence, is not possible to define a degree to which a representations is grounded on an ontological reality.

The concept of a dynamic and collective interpretation of the situation is present as well in the third theory that we are presenting in this paragraph. “Situated cognition”, in opposition to classical representation paradigm, states that context, cultural elements and social frame are the bases of the decision making process. In its view, the behaviour is not based on an abstract algorithm but is strongly bound to the environment. The decision to perform an action is not the consequence of a logical choice taken in accordance with a well predefined plan, but is the consequence of an utilitarian strategy based on the actual context (both physical and social) as perceived in that given moment (Suchman 1987).

The situated cognition paradigm rejects the idea, often adopted by the classical Artificial Intelligence (A.I.) community, that a complete and efficient representation of the environment is the basis of intelligent behaviour (or using the classical representational A.I. motto “in the knowledge is the power”). In real life, actions can be, and often are, taken on the base of a limited set of information that are directly collected in the environment. Such informations are then used to perform actions or to develop partial-plans that still will be flexibly modified in accordance with the new informations collected.

In this vision, which is shared by ethnomethodology, the concept of a priory planning as the basis of human behaviour disappears. The planes are seen as an “a posteriori” rationalization of the taken actions, reconstructed for a reason of coherence. As we will see in chapter five, the situated cognition critics to the A.I. paradigm are at the basis of our multi-user approach that tries to minimalise the use of A.I. in virtual reality.

2.4 A Critic Against The Mind-Body Dichotomy: Enaction,

Coupling and Social Cybernetics

The neglect of the role of the body in human cognition, a consequence of the Descartes distinction between “Res cogitans” and “Res extensa”, have strongly influenced social research in this last year. The most radical adoption of this

(28)

mind-body dichotomy is probably represented by cognitive science. The cognitive approaches to social science like “Theory of Mind” or “Simulation ” theory have been criticized in these last years for the limits, which the exclusion of the embodiment take with it.

To overcome these limitations, a series of approaches that reconsider the role of human body and embodiment in the cognitive process is gaining popularity in the scientific community. One of these embodied paradigms is enaction that considers that knowledge comes through action in the environment. Knowledge is seen not as the consequence of mental manipulation of symbols but as constructed on motor skills, such as the capability to physically manipulate objects. Enaction stressed the role of the organizational properties of the living organisms. One of these primal properties is the autonomy of living organisms and the self-generated identity that distinguish them from the surrounding environment. In such vision, the autonomy is defined by the presence of processes that actively generate and sustain the identity. According to enaction, the identity of living organisms is founded on the principle of operational closure.

The interactions of the emergent entity with its contest give rise to a process of mutual influence that determine the entity-environment coupling (Thompson 2001). Such interaction with the environment is done with the purpose of the continuity of that self-generated identity, and consequently such interaction implies the projections of meanings (and interpretations) on that surrounding environment.

The concept of coupling is primordial in enaction theory and is related with making a relationship between two entities effective. When the coupling is complete the two entity act like a unique structure. A typical example of effective coupling used by various authors is the one of the hammer. When used in a familiar way the hammer becomes a functional extension of the arm in a way that we are not acting with the hammer but we are acting directly onto the nail trough the hammer. In this case we face what is usually defined as an embodied interaction in which we do not interact with a tool but we interact trough a tool which somehow has been incorporated in our system.

Coupling has been studied in different fields but take a particular relevance in the field of cybernetics and in particular on the works of Varela and Maturana that is seen as the basis of enaction theory. Based on their work on biological systems the two authors formulate the hypothesis that cognitive systems are not based on symbolic representation and global controls but are the results of a set of local interactions and local cooperation of relatively autonomous (not supervised) signal elaboration systems. The global behaviour of a system (a living creature but as well a colony of individuality) is then not the consequence of a centralized coordination but is an emergent phenomena due to both the connections that are internal to the system and to the structural coupling of such system with his environment. In their vision the creatures are actually governed by a series of local interaction that take place to various levels form the cellular one to the social one. We can indeed talk about different levels of structural coupling. A structural coupling of first level is the one that concern the interaction between an entity and his environment, a second level coupling is the one that drive the interactions between a creature provided with a

Figure

Figure 3: An avatar involved in a voice chat in Second-life.
Figure 4: An avatar performing a waving-hand animation in Second-life.
Figure 9: Kismet reactions to optimize interaction.
Figure 10: Left: Fireman on the lower balcony proceeds to climb the ladder to reach  the victims (one of which is being suspended over the balcony) on the second floor
+7

Références

Documents relatifs

ces biais entre répondants, militants et adhérents qui fondent ici l’intérêt de ces données, sachant que ces responsables syndicaux ont des attributs essentiels et communs :

Since the over-expression of the three tR- NAs did not affect their thiolation levels (Supplementary Figure S14), we took advantage of this to deconvolute the impact of

Experiments are based on recording brain signals of test subjects via EEG headset during different activities based on moving hand across a table, following a predesignated

The proposed architecture has been used in artistic contexts: for example, (i) a virtual dancer able to dance with a human (who moves on a dance pad) and to adapt to the rhythm of

Key Words: Magic Ring, tracking tool, tracker, virtual world, Second Life, avatar, avatar behaviour, spatiality, methodology, ethnography, Big Data, quali- quantitative data..

Because T lymphocytes implied in cancer and auto-immune diseases are near the Recognition Threshold, they are affected soon when exposure conditions inhibit transitions to state (CR)

Specifically, many recent works have focused on creating 6-DoF VR experiences from 360° photographs [2, 4, 9], demonstrating rendering solutions able to provide motion parallax

Pour discrétiser le problème de contact unilatéral, deux for- mulations sont testées: une avec la méthode de pénalisation et I'autre avec une méthode mixte basée sur