De la navigation bio-inspirée
au développement sensorimoteur d’un robot humanoïde
Philippe Gaussier
ETIS (Equipes de Traitement de l’Information et Systèmes) UMR CNRS 8051, Neurocybernetics team,
Cergy Pontoise University, France
Cognition, brain, neurons...
The limits of a reductionist approach…
- Development
- Social interactions
p2
Primate visual system
(Van Essen 91)
Problem:
Knowing the neuronal activity is not always helpful to understand the cognitive processing !
Understanding the link between the link between the neuronal activity and the behavioral dynamics.
Solution:
Our approach
• Find a minimal model justified by strong theoretical arguments:
– single NN formalism and architecture AND several tasks – Taking into account a minimal number of biological
structures,
– Trying to understand what is brought by a new structure on the global behavior (otherwise do not take it into account), – Using robots to test the behavioral consequences of a given
model (proof by failure!)
Strong collaborations with neurobiologists and psychologists
« Virtual » Laboratory
Joint works with:
Jean Paul Banquet : neurobiological modelling
Bruno Poucet (Marseille 3C): exp. neurobiology + S. Wiener group (Paris) ANR Neurobot
Jacqueline Nadel: developmental psycho pathology (+ inter-lab. association CNRS)
Yann Coello, Yvonne Delevoye (URECA-Lilles) ANR INTERACT / SESAME TINO
• Lola Cañamero (Feelix growing UE STREP project) University of Hertfordshire, UK
+ K. Bard (Portsmouth)
Robotique et neurosciences ou
Robotique et sciences cognitives?
23/06/11 P. Gaussier, ETIS, Cergy Pontoise p7
Equipe neurocybernétique
23/06/11 P. Gaussier, ETIS, Cergy Pontoise p8
Neurobiologie
Expérimentation en exterieure
navigation visuelle / syst neuromimétique
Apprentissage en ligne:
associations sensori-
motrices, carte cognitive et planification
modélisation en neurosciences computationnelles
Perception visuelle / mécanismes attentionels
ETIS - Equipe Neurocybernétique
INTERACTIONS
Psychologie developpementale
Interactions sociales + IHM
Apprentissage par imitation
Modèle des émotions / référencement social Robot hydraulique Interactif
Lien mécanique & cognition (compliance physique)
Modèles dynamiques neuronales dynamiques d’interactions
Systèmes Multi-Agents
- Solutions émergentes - Optimisation collective - Spécialisation des agents
Plan
1. Navigation autonome bio-inspirée 2. Apprendre en interagissant
3. Emotions et interactions sociales
une même architecture neuronale pour des applications très différentes
P. Gaussier, ETIS, Cergy Pontoise p11 23/06/11
1 st part
Navigation bio-inspirée Hipp-BG-PFC loop
Navigation and the building of attraction basin (+ transition building, novelty detection,
frustration…)
Action selection problem
How to deal with the bifurcation points?
Diffusion Ford – Bellman alg.
C
D Goal
Solution: “transitory states”
One transition is always associated to a single action C
D Goal
B
[Revel98,Gaussier01]
[Hok2007]
• Sequence learning: hipp. model Use timing capability of DG/CA3
[Banquet98]
Goal firing
Main field Goal firing
Main field Goal firing
Rat at goal for 2s
Hok et al. (2007)
Analyse de l’activité des cellules de lieu dans la période d’attente au but
Population activity
P. Gaussier, ETIS, Cergy Pontoise p18
Latent learning of the cognitive map.
Return path to the goal place from random
departure points.
Stay for 7s at the goal place.
23/06/11
[Cuperlier 2006,
Hirel 2010]
2d part
Interactions capabilities
P. Gaussier, ETIS, Cergy Pontoise
Immediate imitation
Minimal mechanisms:
- Ambiguity of perception - Homeostatic control
emergence of immediate
imitation [Andry et al 2002 & 2004]
Control of a 4DOF arm
Pan-tilt camera
Sensori-motor learning (robot alone)
Immediate imitation
23/06/11 20
P. Gaussier, ETIS, Cergy Pontoise 21
Sensori-motor controller
learning of visuo-motor associations (online learning)
visual activation of motor attractors
extraction of the command on the basis of the selected attractors [Fukuyori 2008]
Minimal mechanisms necessary to the bootstrap of imitation ?
Observational learning
1. Autonomous learning of visuo-motor associations (babbling)
2. Inhibition of the motor command during human display (but sequence learning) 3. Reproduction of the sequence from one item to the next (first item is either
triggered by human or by robot position)
[IROS2010]
Learning by demonstration more complex tasks
Cognitive map encoding the sequences of transitions
Visual categorisation to generate the goals planning
Categories: proprioceptive states, transition detected and coded as new states (same as nav.)
Addition of the planning capabilities used for navigation (latent cognitive map learning)
Preliminary conclusions
• « Mirror neurons » can emerge from the sensori-motor learning.
• « mirror neurons » may be the consequence of the sensori-motor learning (not the cause!)
• Is imitation so much important for learning?
P. Gaussier, ETIS, Cergy Pontoise p24 23/06/11
3 d part
Emotions and social interactions
A solution to build a developmental sequence bootstrapped from emotional
resonance…
Imitation and social interactions
(J. Nadel et al 2004, Interactive Studies)
(J. Nadel et al 2004, Interactive Studies)
Imitation and social interactions
Imitation and social interactions
(J. Nadel et al 2004, Interactive Studies)
Desired solution:
One possible architecture:
(based only on sensori-motor loops)
A toy problem: How to model baby learning
to recognize facial expressions?
After only 2 min of online learning
(SAB04, Epirob 06, 08)
Social referencing
•
Toward complex behaviors (manipulation, interactions between behaviors at different levels…)
•
Modeling emotions (metacontrol /communication)
Exp. on the devel.
of the social referencing (associate a value to an object)
[Russel97]
……Pierre……
15/02/11 P. Gaussier, ETIS, Cergy Pontoise p32
Social referencing for object manipulation
33 Providing an emotional value to an object
Behavior: take-and-deposit or avoid
Discussion
&
conclusions
P. Gaussier, ETIS, Cergy Pontoise p34
23/06/11
Conclusion
Les aller-retour neurobiologie/psychologie robotique prennent du temps:
temps de boucle 6/7 ans minimum…
(modèles falsifiables / intérêt des doubles fonctions)
collaboration sur du long terme
management délicat / projets sur 3 ou 4 ans (ANR, Europe…)
23/06/11 P. Gaussier, ETIS, Cergy Pontoise p35
Nouveaux outils pour
les sciences cognitives
Tester des modèles du cerveau en conditions réelles !
Page 3!
"#$%&'(&')*! +'! ,-! ./)0$-&/1*'2! +*! $#+3,'! +4)-$/1*'2! +'5! .6#/7! +8-.&/#))'$')&! 9-/&5! %#*:! &#*&'5! ,'5!
-:&/.*,-&/#)5! +*! :#;#&2! )#*5! -<#)5! 0&-;,/2! %#*:! .6-1*'! -:&/.*,-&/#)2! ,'5! 5%0./9/.-&/#)5! +'! &#*&'! ,-! .6-=)'!
+8-.&/#))'$')&!+'%*/5!,'!$#&'*:!>*51*8?!,-!.6-=)'!+'!:0+*.&/#)!1*/!-)/$':-!,8-:&/.*,-&/#)@!
! !"#$%&'(#$)*+ !*,'-+
.$*+/01+
!*,'-+
.(2+/01+ 3)#-&"+ 4&$55(*%-+ 6$#-55-+7+6$8-+ 9)&:'-+
/8;.(""(,-1+ <;8&%#$)*+ =>+
?#(,-5+ <-*8-.-*#+ @+("#$%&A '($"-+
9+!"#$%&A '($"-+
B-(8+C+=-D+
!"#$% &'(% &'(%
% &)(% &*+,(% &*-./0(% &12,(%
% % &3(% &*+,(% &12,(%
B-(84$#%E+ (AB! AB! C'($-7AD! EE! FDEB! FFG2HF! B2BGAD! EIB2JDJ! D! KJ2KL! GK2KI! D2FB!
B-(8<)''+ (DG! DG! C'($-7AD! EE! FDEB! FFG2HF! B2BGAD! ABE2IBG! D! KJ2KL! IK2JE! F2AI!
=-%D4$#%E+ (DB! DB! C'AG! AB! JKIB! HBI2FI! B2AIK! JG2AD! I! FG2DL! EBE2AD! EG2EK!
=-%DF(G+ (HB! HB! C'AG! AB! JKIB! HBI2FI! B2AIK! JG2KI! D! KJ2KL! EBB2FJ! EI2JK!
9)'&.*+
H"&*DF(G+ (DG! DG! C'AG! AB! JKIB! HBI2FI! B2AIK! AJB2FF! I! FG2DL! IB2FD! DH2HI!
9)'&.*<)''I+ (EG! EG! C'AG! AB! JKIB! HBI2FI! B2AIK! EIB2JDJ! D! KJ2KL! KG2HG! AE2EJ!
9)'&.*<)''J+ (EG! EG! C'AG! AB! JKIB! HBI2FI! B2AIK! EIB2JDJ! D! KJ2KL! KG2HG! AE2EJ!
9)'&.*4$#%EI+ (DG! AB2E! C'AG! AB! JKIB! HBI2FI! B2AIK! EIB2JDJ! D! KJ2KL! KG2HG! AE2EJ!
9)'&.*4$#%EJ+ (DG! AB2E! C'AG! AB! JKIB! HBI2FI! B2AIK! EIB2JDJ! D! KJ2KL! KG2HG! AE2EJ!
KE)&'8-"+
KE)&'8-"4$#%E+ (HB! EAB! C'AG! AB! JKIB! HBI2FI! B2AIK! EIG2GJ! D! KJ2KL! KI2KG! AE2HD!
KE)&'8-"F(G+ (EB! HB! C'AG! AB! JKIB! HBI2FI! B2AIK! JG2KI! D! KJ2KL! EBB2FJ! EI2JK!
?'>)G+
?'>)GF(G+ (HB! B! C'AG! AB! JKIB! HBI2FI! B2AIK! JG2AD! I! FG2DL! EBE2AD! EG2EK!
?'>)G<)''+ (EAB! EAB! C'AG! AB! JKIB! HBI2FI! B2AIK! JG2KI! D! KJ2KL! EBB2FJ! EI2JK!
L"$5#+
L"$5#<)''+ (EAB! EAB! C'($-7AD! EE! FDEB! FFG2HF! B2BGAD! ABE2IBG! D! KJ2KL! IK2JE! F2AI!
L"$5#4$#%E+ (GK! GK! C'($-7EF! D2G! EBEBB! EBGF2KK! B2BEEF! EFI2AA! D! KJ2KL! GJ2IE! E2IH!
L"$5#F(G+ (AJ! AJ! C'($-7EF! D2G! EBEBB! EBGF2KK! B2BEEF! ADH2HG! D! KJ2KL! DB2DE! A2BE!
B(*8M$*,-"+ !! !! CM($-7EF! D2G! EBEBB! EBGF2KK! B2BEEF! EEI2EH! D! KJ2KL! JH2AI! B2HE!
B$:F(G+ (IB! GB! C'IB! KB! JDHB! JJH2BF! E2BA! EEE! I! FG2DL! FK2DH! JG2IA!
B$:<)''+ (IB! KB! C'DB! EGB! FGJB! FHI2FF! A2G! HJ! D! JF2BL! FF2IG! AEI2EG!
B$:4$#%E+ (EAB! IB! C'DB! EGB! FGJB! FHI2FF! A2G! HJ! D! JF2BL! FF2IG! AEI2EG!
N*--4$#%E+ (G! EIG! C'DB! EGB! FGJB! FHI2FF! A2G! KK!(!EGB! D! FJ2BL! EED2JG! AHA2GB!
!*D'-4$#%E+ (DG! DG! C'DB! EGB! FGJB! FHI2FF! A2G! HJ! D! JF2BL! FF2IG! AEI2EG!
!*D'-<)''+ (DG! DG! C'DB! EGB! FGJB! FHI2FF! A2G! HJ! D! JF2BL! FF2IG! AEI2EG!
H)-+:$#%E+ ')!+'<! ')!+'<! ')!+'<! ')!+'<! B! B! B! (DA! DA! E2HL! B2BB! B2BB!
!"#$%&'%($')*'$%($"+"+,('-%#.%/*)(&'%$'//"$+%0'%$1(('&%$1)2.'%&'/%"$+'*&/%'.%("/*+*".%3"$*4".+1&'5%
6*7#$'%89!N#+3,'!O0#$0&:/1*'!'&!</5*',!+*!%:'$/':!%:#&#&4%'!C#$'#!
Robotex
Emotions as a way to structure learning
• Learning to recognize facial expression can precede the learning of face / non face recognition !
• The emotion communication/interaction may be used as a way to shape and trigger more and more complex learning when no explicit reinforcement signal can be used (bootstrap).
Importance of social feedbacks
• Taking into account the emotional development opens
new doors for autonomous learning
Videos on: www.etis.ensea.fr/~neurocyber/Videos/
15/02/11 p38
Visual navigation + arm control (SM level)
[Epirob 2010]
Joint work with EPFL
Use of a saliency map to speed-up learning
40
Saliency map Inspired from [Itti & Koch]
