Mapping crop fields: active perception using SLAM and online path optimization

(1)

LAAS-CNRS

/ Laboratoire d’analyse et d’architecture des systèmes du CNRS LAAS-CNRS

/ Laboratoire d’analyse et d’architecture des systèmes du CNRS

Laboratoire conventionné avec l’Université Fédérale de Toulouse Midi-Pyrénées

GIS micro drones: 5th Garden Workshop - July 10, 2018 -

Mapping crop fields: active perception

using SLAM and online path optimization

(2)

LAAS-CNRS

/ Laboratoire d’analyse et d’architecture des systèmes du CNRS 2

UAVs for precision agriculture

(3)

LAAS-CNRS

UAVs for precision agriculture

(4)

LAAS-CNRS

UAVs for precision agriculture

(5)

LAAS-CNRS

Offline Planning

(6)

LAAS-CNRS

Monocular Graph SLAM

(7)

LAAS-CNRS

Monocular Graph SLAM

(8)

LAAS-CNRS

Monocular Graph SLAM Monocular Graph SLAM

Map point position error (Keyframe RMSE is 0.25m)

* Good keyframe position precision

* Worse map quality :

- Keypoint matching errors

- SLAM marginalizes map points - Covariances = too optimistic

--> Wrong error estimate

(9)

LAAS-CNRS

Simulation Framework architecture Simulation Framework architecture

SAPLING

SLAM+error model + CPP

DSAAM IGN ORTHO-HR

database Wind + Turbulences

(10)

LAAS-CNRS

Learning the relative error Learning the relative error

Features NN Weighted

Graph NN

∀ edges

= overlapping image pairs ∀ pair of nodes

= poses Relative

Error Estimate

weights distance

metric

(11)

LAAS-CNRS

Learning the relative error Learning the relative error

0 200 400 600 800 1000 1200 1400 1600 0.35

0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85

RMSE loss evolution during learning

iteration

RMSE loss

First results are promising Ongoing work :

- Better loss function

- Generating large simulation database - Features selection

- Features for prediction - Integration with planning

(12)

Mapping crop fields: active perception using SLAM and online path optimization

Nicolas Holvoet

LAAS RIS Group ENAC

July 10, 2018

1 / 21

(13)

Introduction

Refine to reach full coverage + quality

Mission planning Real time SLAM data

Goal: fully complete the mapping mission within a minimum time Plan goals deduced online from a vision-based SLAM:

• areas which are not covered yet

• areas with bad acquisitions (blur, wind...) and thus low quality

• areas which would strategically reinforce the map (predictive model)

2 / 21

(14)

Field

Polygon, possibly non-convex with non-convex holes

3 / 21

(15)

Dynamic vehicle model: Dubins path

Shortest curve that connects two points with prescribed initial and terminal headings for a constant-speed vehicle with a minimum turning radius (non-holonomic).

4 / 21

(16)

State of the art

Abundant literature for the coverage path planning (CPP) problem, but great variety of subproblems:

• known or unknown environment

• with or without ”obstacles”

• convex or non-convex shapes

• with or without wind

• holonomic or non-holonomic vehicle

• with or without energy/time/... minimization

Common approaches for the coverage of known environments:

• exact cell decomposition (e.g. trapezoidal, Morse...)

• approximate cell decomposition (e.g. grid, treemap...)

5 / 21

(17)

State of the art

Main constraints:

• non-holonomic, fixed wing, constant speed UAV

• Dubins paths

• constant altitude

• wind: hard, depends on the controller

For now, just integrate the speed over the Dubins trajectory

• performance Intuition:

• straight acquisition segments: boustrophedon/lawnmower patterns

• segments’ direction should reflect locally the shape of the field

• holes shouldn’t matter too much

6 / 21

(18)

State of the art

Most promising approach:

• find a good decomposition of the field

• cover each subfield wih parallel segments in the optimal direction

• connect the segments by solving a Generalized Traveling Salesman Problem

7 / 21

(19)

Connecting the segments with a GTSP

8 / 21

(20)

Connecting the segments with a GTSP

9 / 21

(21)

Finding a good decomposition

Reduce the complexity by generating a set of potential ”cut edges”

e.g. convex decomposition with triangles

10 / 21

(22)

Finding a good decomposition

e.g. extend the edges at reflex (concave) vertices

11 / 21

(23)

Finding a good decomposition

all together

12 / 21

(24)

Finding a good decomposition

Idea: find the best combination of cut edges

Scoring function: estimated duration of the mission with this decomposition

Complexity: roughly 2ⁿ with n the number of potential cut edges Approaches:

• exhaustive (optimal but can be extremely slow, e.g. dynamic programming)

• greedy (fast but suboptimal)

• metaheuristics (in between)

13 / 21

(25)

Encoded decomposition

Decomposing loop

Decoding Scoring

generate a neighbor

(bit ipping) accept/reject

the move 0 0 1 0 ... 1 0 1

Simulated Annealing

1: i ^th cut edge activated

optimal directions _k estimated duration

stopping?

P₁ P₂ P₃

generation of potential cut edges

best decomposition (P_k) with optimal directions (_k)

no yes

polygons P_k

₁⁼

0°

₂⁼

90°

segments generation eld

Segments connector

optimal tour actual duration GLKH

4 2 1

3 5

6

7 9 ^GTSP 8

14 / 21

(26)

Results

(a) (b)

15 / 21

(27)

Results

(c) (d)

16 / 21

(28)

Results

(e) (f)

17 / 21

(29)

Results

(g) (h)

18 / 21

(30)

Results

(i) (j)

19 / 21

(31)

Results

(k) (l)

West wind, 12 m/s (UAV speed: 15 m/s)

20 / 21

(32)

What’s next?

Online replanning

21 / 21