Topological navigation using sensory memory: SLAN versus SLAM

(1)

HAL Id: hal-02468625

https://hal.inria.fr/hal-02468625

Submitted on 6 Feb 2020

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Topological navigation using sensory memory: SLAN

versus SLAM

Philippe Martinet, J. Courbon, Y. Mezouar, B. Thuilot

To cite this version:

Philippe Martinet, J. Courbon, Y. Mezouar, B. Thuilot. Topological navigation using sensory

mem-ory: SLAN versus SLAM. Journée SLAM, Dassault-Aviation, Dec 2009, Saint-Cloud, France.

�hal-02468625�

(2)

Philippe Martinet

Topological navigation using sensory memory: SLAN versus SLAM

LASMEA, Blaise Pascal University, Clermont-Ferrand

Topological navigation using sensory memory

SLAN versus SLAM

Professor Philippe Martinet

J. Courbon, Y. Mezouar, B. Thuilot

LASMEA Laboratory

Blaise Pascal University

martinet@lasmea.univ-bpclermont.fr

http://wwwlasmea.univ-bpclermont.fr/Control

http://robots.lasmea.univ-bpclermont.fr

Dassault Aviation, Saint-Cloud, France, December 16

th

₂₀₀₉

Program of the journey

2

LASMEA

P. Martinet

Navigation toplogique par mémoire sensorielle: SLAN

vs SLAM

12H

IGN/MATIS

N. Paparoditis

16H

Inria-Sophia

Antipolis

P. Rives

Outils et méthodes d'estimation pour le SLAM

-Application au Slam visuel

14H

ENSTA

ParisTech/UEI Lab

D. Filliat

Détection visuelle de fermeture de boucle et applications

au SLAM métrique et topologique

15H

Dassault Aviation

B. Patin

Cloture du séminaire

17H

LAAS

Juan Sola

Initialisation immédiate d-amers en SLAM monoculaire.

Points et lignes

11H

Inria-Sophia

Antipolis

P. Rives

Les enjeux et problématiques du SLAM (Survey)

10H15

Dassault Aviation

B. Patin

Accueil et présentation de la journée

10H

Outline of the presentation

Introduction

• LASMEA

• Sensor based Control

• Navigation

• SLAN vs SLAM

Togological navigation system

• Global system

• Case of vision

• Differents Modules

• SOVIN

Discussion

• Learning process

• Updating process

• Closing the loop problem

A generic model

• A unified model for camera

• Case of fisheye cameras

• Partial euclidian reconstruction

Illustrations

• Indoor applications

• Outdoor applications

3 Philippe

Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

Lasmea

400km south of Paris

200km West of Lyon

_{Mixed Unit 6602}

CNRS/UBP

LASMEA

Universitary Campus

France

Clermont-Ferrand

Director : Michel Dhome

4

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

Lasmea

400km south of Paris

200km West of Lyon

_{Mixed Unit 6602}

CNRS/UBP

Director : Michel Dhome

MATELEC

¾Optoelectronics,

microelectronics

¾Electromagnetism

¾Gaz sensor

GRAVIR

¾Perception System

¾Computer that See

¾ROSACE

33 teacher/researcher

1 research scientist

25 Phd

25 teacher/researcher

2 research scientist,

49 Phd, 2 post-doc

5 Philippe Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

6

Lasmea : GRAVIR-PERSYST

J.P. Derutin

Perception System

PERception SYSTems

Multisensor

Perception

Data fusion

Scene understanding

SLAM

Smart Sensors

Algorithm Architecture Adequation

_{Cognitive Vision}

Architecture and

methods

Parallelism in vision

High speed prototyping tools

Tools for embedded applications

(3)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

Lasmea : GRAVIR-COMSEE

T. Chateau

Artificial Vision

COMputer that SEE

Geometry for visual

perception

Metrology by vision

Automatic rigid scene reconstruction

Vision in Deformable

Environments

Modeling and algorithm for deformable

environment

Visual

Recognition

Real time object tracking

Object recognition

http://comsee.univ-bpclermont.fr

7 Philippe

Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

8

Lasmea : GRAVIR-ROSACE

P. Martinet & Y. Mezouar

ROSACE

ROboticS and Autonomous Complex systEms

VISIR

VIsual ServoIng of Robots

Omnidirectional visual servoing

Topological navigation through sensory memory

Multi-sensor-based control

AGV

Automatic Guided Vehicles

Control design under uncertain dynamics

Hybrid control architecture

Multi-robot-system control

MICMAC

Modeling, Identification and

Control of Complex Machines

Vision-Based Control of Parallel Robots

Control of redundant robots

Control of High Dynamics System

http://wwwlasmea.univ-bpclermont.fr/Control http://robots.lasmea.univ-bpclermont.fr

400Hz

1kHz

…

15 to 60Hz

…

15 to 60Hz

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

9

Lasmea : GRAVIR

People involved in SLAM or near SLAM problems

PERSYST

F. Berry, R. Chapuis, F. Chausse, P. Checchin, J.P. Derutin, L.

Trassoudaine

(IMPALA/CITYVIP)

(CITYHOME/EURIPIDE/BRI)

COMSEE

M. Dhome, M. Lhuillier , E. Royer

(MOBIVIP/CITYVIP/BODEGA)

(FACT/CITYHOME/EURIPIDE/BRI)

ROSACE

Y. Mezouar, P. Martinet, B. Thuilot

(MOBIVIP/CITYVIP/BODEGA/R-DISCOVER/ARMEN)

(SAFEMPOVE, FACT/CITYHOME/EURIPIDE/BRI/)

Autonomous navigation (RTK-GPS, Vision, Multi-sensor)

Platoon: Autonomous leader (RTK-GPS, Vision, Multi-sensor)

Platoon: Leader manually driven (RTK-GPS, Vision)

2D/3D maps building (vision, Radar, Range finder, Multi-sensor)

Applications

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

10

Sensor based control: History

˜

y

˜θ

M

= (

s, C

(s)

)

d

0

d

1

d

2

d

5

d

4

d

3

Lateral control in cartesian space:

Lateral control in sensor space:

Bringing

³

y, ˜

˜

θ

´

to (0, 0)

∆

Explicite declaration of ∆ and

required cartesian localization

d =(d

0

, d

1

, d

2

, d

3

, d

4

, d

5

)

T

Bringing d to d

∗

No explicite declaration of ∆ and

do not required cartesian localization

d

∗3

d

∗ 4

d

∗₅

d

∗0

d

∗ 1

d

∗ 2 [ESPIAU87]

Philippe Topological navigation using sensory memory: SLAN versus SLAM

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

11

Sensor based control: Basic concepts

x

y

z

Fo

x

y

z

Fc

x

y

z

Fa

x

y

z

Fc

Case of embedded camera

ω

υ

x

: 3D pose

If

(fixed object)∂s_∂t

= 0

˙s = L

s

v

Interaction matrix

˙s = L

s

v

+

∂s

_∂t

s

= s(x, t)

˙s =

∂s

∂x

dx

dt

+

∂s

∂t

˙s =

∂s

∂x

v

+

∂s

∂t

[SAMSON90,ESPIAU92]

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

12

Navigation

Planning

Mission, route

Localization

Single, platoon mode

Obstacle

avoidance

Scheduling

Execution level

Control

FDIR

monitoring

End user level

Reference trajectory

Modified reference trajectory

(4)

Introduction

model

Topological

navigation system

Illustrations

Discussion

13

Navigation

Navigation scheme

Planning

Mission, route

Obstacle

avoidance

Control

Localization

Reference trajectory

Scheduling

Execution level

FDIR

monitoring

End user level

Introduction

model

Topological

navigation system

Illustrations

Discussion

14

Navigation

3D Map based navigation (absolute navigation)

Online step

Exteroceptive

sensors

Proprioceptive

sensors

Itinerary

Selection

& Execution

Localization

GIS 3D map based Reference trajectory Global Lateral deviation Angular deviation Curvature Curvilinear abscissa Philippe Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

15

Navigation

Learning step

Exteroceptive

sensors

Proprioceptive

sensors

GIS

Itinerary

Selection

& Execution

Topological

Indexation

Extraction

of features

_{Augmented GIS}

Topological

Representation

sensory memory

Memory based navigation (relative navigation)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

16

Navigation

Memory based navigation (relative navigation)

Online step

Exteroceptive

sensors

Proprioceptive

sensors

Itinerary

Selection

& Execution

Localization

Augmented GIS Topological Representation Reference trajectory LOCAL Lateral deviation Angular deviation Curvature Curvilinear abscissa Philippe Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

17

SLAN vs SLAM

Sensor space

Cartesian (relative)

Cartesian

Control space

Relative

Absolute

Localization

Topological

Semi-Metric

Metric 2D/3D

MAPS

SLAN

SLAM

Outline of the presentation

Introduction

• LASMEA

• Sensor based Control

• Navigation

• SLAN vs SLAM

Togological navigation system

• Global system

• Case of vision

• Differents Modules

• SOVIN

Discussion

• Learning process

• Updating process

• Closing the loop problem

A generic model

• A unified model for camera

• Case of fisheye cameras

• Partial euclidian reconstruction

Illustrations

• Indoor applications

• Outdoor applications

(5)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

19

A unified model for camera

Single view point system :

Classification

F’ F F F’

Planar

Elliptical

F

Parabolic

F’ F

Hyperbolic

Log/polar retina

[Geyer00]

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

20

A unified model for camera

F’ F F’

Planar

Elliptical

F

Parabolic

F’ F

Hyperbolic

Mirror type

Parabolic

1 1+2p

Hyperbolic

Elliptical

Planar

0

1 Conventional

0

1 ϕ

ϕ and ξ : Mirror parameters

ξ

d+2p

√

d

2

+4p

2

d

−2p

√

d

2

_+4p

2

d

√

d

2

+4p

2

d

√

d

2

_+4p

2

Single view point system :

Classification

_[Geyer00]

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

21

A unified model for camera

m

F

c

F

Mirror : unitary sphere

ξ

m3D(X, Y, Z)

Image plane

m

p

m

n

K

M

m

=

1ρ

⎡

⎣

X

Y

Z

⎤

⎦

ρ =

kmk =

√

X

2

+ Y

2

+ Z

2

m

p

= K

M

m

n

m

n

= [x

T

β]

T

=

⎡

⎢

⎣

X

Z + ξρ

Y

Z + ξρ

β

⎤

⎥

⎦

Single view point system :

case of point

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

22

A unified model for camera

Generic Projection function

m F c F

mirror

Image plane

Special cases :

Î perspective projection

Î spherical projection

ϕ and ξ : Mirror parameters

ϕ - 2 ξ

ξ

m

p

M

=

⎡

⎣

ϕ

− ξ

0 ϕ

_{− ξ 0}

0

1 ⎤

⎦

K

=

⎡

⎣

f

0 f s

f r

u

v

0

1 ⎤

⎦

m3D(X, Y, Z)

m

p

= K M

_{| {z }}

f(m

3D

)

K

M

m

² = 1 and ξ = 0

² = 0 and ξ = 1

f(m

3D

) =

⎡

⎢

⎣

X ²Z+ξ√X2_+Y2_+Z2 Y ²Z+ξ√X2_+Y2_+Z2

1 ⎤

⎥

⎦

Single view point system :

case of point

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

23

Case of fisheye camera

Fisheye camera model

:

definition

Pinhole camera

Fisheye camera

F

r = distance between X and the image point

F

θ = angle between the incoming ray and the principal axis

F

Fm

: camera frame

F

Fi

: image frame

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

24

Case of fisheye camera

Fisheye camera model

:

Pinhole based models

one parameter

n parameters

F

r

1 f

(r

p

) = r

p

L(r

p

, n)

F

r

2 f

(r

p

) =

_L(r

r

p p

,n)

F

r

f

3 (r

p

) =

r

p

1+k

1

r

2p

Mapping :

[T. Pajdla97,Hartley00,Zhang98,Ma2006…]

[Fitzgibbon01]

(6)

Introduction

model

Topological

navigation system

Illustrations

Discussion

25

Case of fisheye camera

n1+n2 parameters

F

r

4 f

(r

p

) = r

p

L

_L

1

(r

p

,n

1

)

2

(r

p

,n

2

)

F

r

5 f

(r

p

) = s log(1 + λr

p

)

F

r

f

(r

p

) = λ

r

p

F

r

6 f

(r

p

) =

ω

1 arctan

¡

2r

p

tan

ω

2 ¢

s is a scaling factor and λ a gain to control the amount of distortion

The ”distortion function” λ =

{λ1

, λ

2

, . . . λ

n} can be then fitted by a

para-metric model

[Hartley07]

[Li05]

[Basu95]

[Devernay01]

Fisheye camera model

:

Pinhole based models

Mapping :

Introduction

model

Topological

navigation system

Illustrations

Discussion

26

Case of fisheye camera

Fisheye camera model

:

Captured rays based models

Mapping :

F

r

1 f

(θ) = f θ

F

r

3 f

(θ) = f sin θ

F

r

2 f

(θ) = 2f tan

¡

θ

2 ¢

F

r

4 f

(θ) = f sin

¡

θ

2 ¢

cameras with limited distortions

[Kingslake89]

stereographic projection

preserves circularity and thus project 3D

local symmetries onto 2D local symmetries

[Fleck94,Stevenson95]

orthogonal or sine law projection

[Ray94]

equisolid angle projection

[Smith92]

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

27

Case of fisheye camera

Fisheye camera model

:

Captured rays based models

Mapping :

F

r

5 f

(θ) = f (k

1 θ + k

2 θ

3 +

· · · + k

n

θ

2(n

−1)+1

)

F

r

6 f

(θ) = α sin(βθ)

F

r

7 f

(θ) = a tan(θ/b) + c sin(θ/d)

Improvment of the polynomial model accuracy

[xiong97,kannala04,schwalbe05,Scaramuzza06]

[Kumler00]

F

α : scale factor

F

β : radial mapping parameter

[Bakstein02]

Combination of

- stereographic projection (with parameters a, b)

- equisolid angle projection (with parameters c, d)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

28

Case of fisheye camera

F

c

: frame attached to the conventional camera

F

Fm

: frame attached to the unitary sphere

Optical

center

Principal

Projection

center

Unitary

sphere

X = [X Y Z]

T

_{x = [x y 1]}

T

Step 1 : Projection on the unitary

sphere

_X

m

in F

m

: X

m

= X/ρ

where ρ =

kXk =

√

X

2

_{+ Y}

2

_{+ Z}

2

Step 2 : Perspective projection

on the normalized image plane Z = 1

− ξ

x’=f(X) =

∙

_X

εsZ + ξρ

Y

εsZ + ξρ

1 ¸>

spherical projection : ε

s

= 0 and ξ = 1

Catadioptric model : εs

= 1 and ξ = 1

pinhole model : ε

s

= 1 and ξ = 0

Generic camera model

:

Unified Spherical Model

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

29

Case of fisheye camera

Generic camera model

:

Unified Spherical Model

F

Fc

: frame attached to the conventional camera

F

m

: frame attached to the unitary sphere

Optical

center

Principal

Projection

center

Unitary

sphere

X = [X Y Z]

T

_{x = [x y 1]}

T

Step 3 : Projection onto image plane

x = M x’

M =

⎛

⎝

f κ

0

0 δf κ

0

1 ⎞

⎠ δ is equal to ±1

κ > 0

f : focal length

perspective camera : κ = 1 and δ = +1

catadioptric cameras : δ =

−1

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

30

Case of fisheye camera

Generic camera model

:

Unified Spherical Model

F

Fc

: frame attached to the conventional camera

F

m

: frame attached to the unitary sphere

Optical

center

Principal

Projection

center

Unitary

sphere

X = [X Y Z]

T

_{x = [x y 1]}

T

Step 4 : Pixel transformation

m = K x

plane-to-plane collineation K

K =

⎛

⎝

k

u

s

uv

u

0

0 k

v

0

1 ⎞

⎠

(u0, v0)

T

_{: position of the optical center}

k

u

and k

v

: scaling along the x and y axes

s

uv

: skew

setting ξ = 0, κ = 1 and δ = +1

½

x

p

= f

p

X/Z

y

p

= f

p

Y /Z

Pinhole camera

(7)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

31

Case of fisheye camera

Generic camera model

:

Unified Spherical Model

For fisheye camera

r

f

= r

f

(θ) =

_1+ξf√f_tantan θ2_θ+1

is a T

2

-mapping linking the radius r

f

and the incidence angle θ

r

f

= r

f

(r

p

) =

ff fprp 1+ξ s r2p f 2p+1

Constraint 1 : easily verified (r

f

(0) = 0)

Constraint 2 : r

f

(k) is monotonically increasing for k > 0

is a T

1

-mapping linking the perspective Radius

rp

and the fisheye radius rf

The generic camera model is candidate

for fisheye camera modeling

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

32

Case of fisheye camera

Validation with calibration toolbox [MEI07]

Computation of f

f

and ξ (Producers data)

Model fitting for all existing fisheye cameras

Calibration

List of fisheye cameras considered

List of models considered

Polynomial model using perspective projection

Polynomial model using unified model projection

Proposed unified model

r

f

(r

p

) = r

p

(1 + a

1

r

2p

+

· · · + a3

r

6p

)

r

f

(r

u

) = r

u

(1 + a

1

r

2u

+

· · · + a3

r

u6

)

r

f

= r

u

The generic camera model is validated with an average

of 0.2 pixels reprojection error for fisheye camera modeling

Only one parameter required

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

33

Partial euclidian reconstruction

> 0

Introducing

where

γ

_x

=

p

1 + (1

− ξ

2 )(x

2 + y

2 )

with

m

n

Unified model - Case of points

η = s

ρ |Z|

=

p

1 + X

2

_/Z

2

_{+ Y}

2

_/Z

2 s=sign(Z

)

x =

_1+ξηX/Z

y =

_1+ξηY /Z

(1 + ξη) x =

X Z

(1 + ξη) y =

Y Z

η

2

_{− (x}

2

_{+ y}

2

_{)(1 + ξη)}

2

_{− 1 = 0}

η =

±γ

x

+ξ(x

2 +y

2 )

1−ξ

2 _(x

2 _+y

2 ₎

η computed without ambiguity

m

= (η

−1

+ ξ)m

m

= [x y

_1+ξη

1 ]

T

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

34

Partial euclidian reconstruction

mirror mirr_or

m

F

m

F

c

m

_∗ m

F

∗ m

F

∗ c

m

n

m

_∗ n

m

π

(R,t)

Reference plane π in F

∗ m

[X Y Z]

T

_m

_{coordinates in}

_F

m

[X

∗

Y

∗

Z

∗

]

T

_m

_{coordinates in}

_F

∗ m

d

∗

n

∗

π

∗T

_{= [n}

∗T

_{− d}

∗

_]

Unified model - Case of points

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

35

Partial euclidian reconstruction

Homogeneous coordinates:

m

= [X Y Z H]

T

_{and m}

∗

_{= [X}

∗

_Y

∗

_Z

∗

_H

∗

_]

T

The distance from the world point m to the plane (π)

m

= (η

−1

+ ξ)m =

1ρ

£

X

Y

Z

¤T

m

∗ m

= (η

∗−1

+ ξ)m

∗

=

ρ1∗

£

X

∗

_Y

∗

_Z

∗

¤T

m

= [x y

_1+ξη

1 ]

T

m

∗

_{= [x}

∗

_y

∗

1 1+ξη

∗

]

T

ρ(η

−1

+ ξ)m =

£

I

3

0 ¤

m

=

£

R

t

¤

m

∗

d(m, π) = π

∗T

_{· m}

∗

_{= [n}

∗T

_{− d}

∗

_][X

∗

_Y

∗

_Z

∗

_H

∗

_]

T

_{= ρ}

∗

_(η

∗−1

_{+ ξ)n}

∗T

_m

∗

₋

d

∗

_H

∗

Unified model - Case of points

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

36

Partial euclidian reconstruction

where

b

∗ π

=

h

0

_1×3

−

d(m,π)d∗

i

A

∗ π

=

h

I

3 n ∗ d∗

i

T

Euclidian Homography

with H

π

= R +

d

t

∗

n

∗T

and α =

−

d(m,π)

d

∗

H

∗

₌

ρ

∗

_(η

∗−1

_+ξ)

d

∗

n

∗T

m

∗

−

d(m,π)

d

∗

ρ

∗

_(η

∗−1

_{+ ξ)m}

∗

₌

£

_X

∗

_Y

∗

_Z

∗

¤T

m

∗

_{= ρ}

∗

_(η

∗−1

_{+ ξ)A}

∗

π

m

∗

+ b

∗

π

ρ(η

−1

_{+ ξ)m =}

£

_R

_t

¤

_m

∗

ρ(η

−1

_{+ ξ)m = ρ}

∗

_(η

∗−1

_{+ ξ)H}

_π

_m

∗

_{+ αt}

(8)

Introduction

model

Topological

navigation system

Illustrations

Discussion

37

Partial euclidian reconstruction

Unified model - Case of points

H

π

R

and t

d∗

=

_dt∗

m

= β

x,x

∗

H

π

m

∗

Linear form

m

× H

π

m

∗

= 0

where β

x,x

∗

=

η

∗−1

_+ξ

η

−1

+ξ

ρ

∗

ρ

σ =

_ρ

ρ

∗

= (1 + n

∗T

R

T

t

d

∗

)

(η

∗−1

_+ξ)n

∗T

_m

∗

(η

−1

+ξ)n

∗T

R

T

_m

m

_{∈ π}

Introduction

model

Topological

navigation system

Illustrations

Discussion

38

Partial euclidian reconstruction

Unified model - Case of lines

Similar results are obtained from lines

(using polar lines)

m F c F Image plane ξ

0

1

2

5 4 5 3 5 2 2 5 1 2 5 0

+

_y

+

=

A

x

A

xy

A

y

A

x

A

(

)

5

,

A

K

h

B

i M i

ξ

=

Plücker coordinates

L

Unitary sphere

u : unitary vector along the line L

L = (u, h, h)

T

h : unitary vector orthogonal

to the interpretation plane

the frame and the line

h : distance between the origin of

h

u

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

39

Partial euclidian reconstruction

Unified model - Case of lines

Similar results are obtained from lines

Polar lines

A

A’

Conic Φ

l

i

∝ ΦA

plane of φ

A a point in the definition

φ a 2D conic curve

Polar line l

l

i

: the corresponding polar line

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

40

Partial euclidian reconstruction

Unified model - Case of lines

Similar results are obtained from lines

Polar lines

m F c F ξ ur nr

L

x

i

T

K

−T

ΩK

−1

x

i

= 0

Ω

i

The polar line of the optical center with respect to the conic Ω

i

is given by:

Ω =

⎛

⎝

h

2 x

− ξ

2

¡

1 − h

2 y

¢

h

x

h

y

¡

1 − ξ

2

¢

h

x

h

z

h

x

h

y

¡

1 − ξ

2

¢

_h

2 y

− ξ

2

¡

1 − h

2 x

¢

h

y

h

z

h

x

h

z

h

y

h

z

h

2z

⎞

⎠

l

i

∝ Ω

i

O

i

_O

i

= [u

0

v

0

1]

T

= K[0 0 1]

T

principal point

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

41

Partial euclidian reconstruction

Unified model - Case of lines

Similar results are obtained from lines

m F c F Image plane ξ

h

L

Unitary sphere ξ

Virtual image plane

Corollary:

The polar line l

i

computed from the

physical conic curve projection of L in the

omnidirectional image is the perspective

projection of L into the virtual camera

image plane.

l

i

∝

K

−T

ΩK

−1

O

i

∝ K

−T

_ΩK

−1

_K

⎛

⎝

0

1 ⎞

⎠

∝

K

−T

⎛

⎝

h

xy

h

z

⎞

⎠

u

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

42

Partial euclidian reconstruction

Unified model - Case of lines

Reference plane π in F

∗ m

π

∗T

_{= [n}

∗T

_{− d}

∗

_]

[u

∗

_{, h}

∗

_{, h}

∗

_{] L coordinates in}

_F

∗ m

[u, h, h] L coordinates in

Fm

(9)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

43

Partial euclidian reconstruction

Unified model - Case of lines

Similar results are obtained from lines

X1

and

X2

are two points in the 3D space lying on the line

L

r =

h

∗

= (1 + t

∗

d

>

R

>

n

∗

)

kn

∗

_×K

>

_l

∗ i

k

kRn

∗

_×K

>

l

i

k

li

× G

−>

_l

∗

i

= 0

with 4 couples (li, l

∗ i

)

l

i

∝ G

−>

l

∗i

R

and t

d∗

=

_dt∗

Linear form

l

i

∝ K

−T

H

−T

K

T

l

i∗

G

= KHK

−1

H

= R +

t d∗

n

∗T

l

i∗

∝ K

−T

h

∗

l

i

∝ K

−T

h

l

i

∝ K

−T

H

−T

h

∗

h

∝ H

−>

_h

∗ Philippe Martinet

Outline of the presentation

Introduction

• LASMEA

• Sensor based Control

• Navigation

• SLAN vs SLAM

Togological navigation system

• Global system

• Case of vision

• Differents Modules

• SOVIN

Discussion

• Learning process

• Updating process

• Closing the loop problem

A generic model

• A unified model for camera

• Case of fisheye cameras

• Partial euclidian reconstruction

Illustrations

• Indoor applications

• Outdoor applications

44 Philippe Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

45

Global system

sensoriel Chemin Carte topologique niveau 1 Carte topologique niveau 0 Objectif courante Image Image courante LOCALISATION Image cible intermediaire Memoire sensorielle Phase hors−ligne Carte sensorielle Situation Commande courante COMMANDE CARTOGRAPHIE Sequences acquises de chemin Recherche lors de l’apprentissage Phase en−ligne Philippe Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

46

Global system

sensoriel Chemin Carte topologique niveau 1 Carte topologique niveau 0 Objectif courante Image Image courante LOCALISATION Image cible intermediaire Memoire sensorielle Phase hors−ligne Carte sensorielle Situation Commande courante COMMANDE CARTOGRAPHIE Sequences acquises de chemin Recherche lors de l’apprentissage Phase en−ligne

Supervised learning step: building of the sensory memory

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

47

Global system

Initialization: primary localization

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

48

Global system

(10)

Introduction

model

Topological

navigation system

Illustrations

Discussion

49

Case of vision

sensoriel Chemin Carte topologique niveau 1 Carte topologique niveau 0 Objectif couranteImage Image courante LOCALISATION Image cible intermediaire Memoire sensorielle Phase hors−ligne Carte sensorielle Situation Commande courante COMMANDE CARTOGRAPHIE Sequences acquises de chemin Recherche lors de l’apprentissage Phase en−ligne Philippe Martinet

Introduction

model

Topological

navigation system

Illustrations

Discussion

50

Sensory map (CS)

9Initial localization

9Sensor based control

9Vizualization of the environment

9Human interface

Topological map level 1 (CT1)

9Image network representation

9Navigability

9Commandability

9Plannification

Topological map level 0 (CT0)

9High level representation : corridor,

routes, ..

Case of vision

Sensory memory

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

51

Different modules

Key images selection

Building the sensory memory

Learning step (first arc, first two nodes)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

52

Different modules

Learning step (expanding the processus…)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

53

Differents modules

Initial localization

How to find the closest image in the database ?

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

54

Differents modules

Initial localization

(11)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

55

Differents modules

Initial localization

How to find the closest image in the database ?

Use of a regular mesh

[PERSSON04]

Cubic interpolation of the image surface

Global descriptor :

vector of the interpolated grey level

pixels at the position of the control point

Control point distribution (one example)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

56

Differents modules

Initial localization

How to find the closest image in the database ?

Local descriptor :

interest points detected by

Harris Stephen detector

Use of patches 11x11

Matching done by using the centered cross correlation score ZNCC

Use of the number of matched point to compute the similarity between 2 images

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

57

Differents modules

How to find the closest image in the database ?

Evaluation of the approach [ICRA08] using three image databases regarding 3 main criteria:

Used memory size, percentage of success, computation time for localization

Several approaches were compared : GONZ [GONZALES02], PHLAC [LINAKER04],

CUB, SURF, SIFT, HARRIS, CUBHAR

Initial localization

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

58

Differents modules

How to find the optimal path ?

Planifying the visual route

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

59

Differents modules

How to find the optimal path ?

Planifying the visual route

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

60

Differents modules

How to find the optimal path ?

(12)

Introduction

model

Topological

navigation system

Illustrations

Discussion

61

Differents modules

Epipolar constraint can be expressed by :

For pinhole model

Scaled euclidian reconstruction

From five couples of points

E can be estimated

Outliers are rejected by

using RANSAC

[Nister04]

X

m

X

∗ m

m

p

m

∗ p

One line localization

Introduction

model

Topological

navigation system

Illustrations

Discussion

62

Differents modules

State estimation and control (one example)

F

Fi

= (Oi, Xi, Yi, Zi)

F

Fi+1

= (O

i+1

, X

i+1

, Y

i+1

, Z

i+1

) the frames attached to the robot when

Ii

State model of the mobile robot

Kinematic model of the mobile robot

(s, y, θ)

Chained

System

theory

[IAV04]

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

63

Differents modules

State estimation and control (one example)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

64

SOVIN

A global software environment for topological navigation in wide spaces

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

65

SOVIN

A physical database organized in three levels

Physical database

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

66

SOVIN

An efficient database interface for on line and real time applications

(13)

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

67

SOVIN

A generic library (generic visual sensor) for visual algorithm processing

Generic image

processing library

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

68

SOVIN

An efficient Human Machine Interface for managing applications

HMI

Outline of the presentation

Introduction

• LASMEA

• Sensor based Control

• Navigation

• SLAN vs SLAM

Togological navigation system

• Global system

• Case of vision

• Differents Modules

• SOVIN

Discussion

• Learning process

• Updating process

• Closing the loop problem

A generic model

• A unified model for camera

• Case of fisheye cameras

• Partial euclidian reconstruction

Illustrations

• Indoor applications

• Outdoor applications

69 Philippe

Martinet

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

70

Applications and projects in LASMEA

AGV using visual memory

04

05

06

07

08

09

10

11

03 AGV

Project

02 C

la

ss

ic

al

ca

m

er

a

(I

A

V

04 )

O

m

ni

di

re

ct

io

na

l

(I

R

OS

05 )

Fi

sh

ey

e

(1

,7

km

)(

IC

A

R

-C

V

08 )

WACIF OMNIBOT BODEGA CITYVIP

SAFEMOVE FACT CITYHOME

µDrone

W

id

e

an

gl

e

(I

R

O

S0

9)

O

m

ni

di

re

ct

io

na

l

(I

C

R

A

07 )

C

la

ss

ic

al

ca

m

er

a

(I

C

R

A

05 )

BRI R-DISCOVER ARMEN

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

71

Indoor applications : WACIF project (2002-2005)

Autonomous Robot for Telepresence navigation, localization, learning, warning

and communication capabilities

¾

Study and development of a demonstrator like a personal

robot fully integrated in the context of « wireless home »

¾

An innovant domotic HMI dedicated for software useful

for human, family and home applications

¾

Telepresence and telesurveillance tasks

home supervision through any wireless point, abnormal

event detection in family environment (intrusion, noise, …)

Autonomous navigation strategy

Wireless communication and services

Introduction

A generic

model

Topological

navigation system

Illustrations

Discussion

72

Indoor applications : WACIF project (2002-2005)

Global navigation strategy

Room1

Room2

Topological representation

Predefined trajectory

Graph representation

Visual memory concept

Room3

Starting point

Ending point

HS = HOME Space

RS

i

= Room Space i

HS =

P

_i

RS

i

Hypothesis : knowledge

Topological representation step

is a part of the knowledge

we get from the

HOME environment