Modèles graphiques stochastiques et optimisation pour la gestion des réseaux écologiques

(1)

Modèles graphiques stochastiques et optimisation pour la gestion des

réseaux écologiques

Nathalie Peyrard et Régis Sabbadin

Unité de Mathématiques et Informatique appliqués, Toulouse (MIAT)

INRA-ONIRIS,

(2)

Conservation of multiple species in food webs

On which species should we spend our money if our goal is to preserve biodiversity?

Conservation?

03/06/2015 INRA-ONIRIS, Nantes Régis Sabbadin

(3)

Conservation of multiple species in food webs

On which species should we spend our money if our goal is to preserve biodiversity?

Conservation?

03/06/2015 INRA-ONIRIS, Nantes

(4)

Spatial sampling of weeds for map reconstruction

How can we choose adaptively the locations to sample in

order to reconstruct a “reliable” weed map?

(5)

Collective management of crops resistance to pathogens

Where and when should we allocate

Resistant crops/protection actions?

(6)

These problems are ecological management problems

 Choosing and applying conservation actions!

Conservation

(7)

These problems are ecological management problems

 Choosing and applying conservation actions!

 Choosing (adaptively) weed sample locations

(8)

These problems are ecological management problems

 Choosing and applying conservation actions!

 Choosing (adaptively) weed sample locations

 Allocating crop systems in space/time

(9)

These problems involve networks

(10)

These problems involve uncertainty

 Threatened species persistence is uncertain!

 Weeds (especially the seed bank) are barely detectable!

 Pathogens dynamics/spread are uncertain!

(11)

Artificial Intelligence tools for

ecological networks management

 AI tools for the

Management of stochastic processes on networks

 These tools are based on Stochastic graphical models



Bayesian networks



Markov Random Fields



(Factored) Markov Decision Processes

 Plus the use of optimization/approximation methods



Dynamic programming



Reinforcement learning



Continuous optimization



Heuristics

(12)

Description of three applications

 Applications in Ecology / Agro-ecology

 Biodiversity conservation in Food webs

 Optimal weeds sampling for map reconstruction

 Management of ecological processes / crop pests

(13)

Application I : Conservation of multiple species in food webs

 Problem : Multiple species protection, within a network of trophic relations

 Management decision : which species should we protect (and when) , in order

Eve MacDonald-Madden & Iadine Chadès (CSIRO and University of Queensland) Peter Baxter, William Probert & Hugh Possingham (University of Queensland)

Edward Game (The Nature Conservancy) Nathalie Peyrard & Régis Sabbadin (INRA-MIAT)

(14)

 Classically, trophic relations in food webs are:

 deterministic, qualitative or quantified by mass flows (dynamical systems)

 The Bayesian network approach is:

 stochastic and quantified by conditional probabilities of presence

Food webs and bayesian networks

present/absent = 1/0

Absent Present

Present

P(HS,OF,CF) = P(HS|OF,CF) P(OF) P(CF)

1 1..

( ,..., _n) _i( | Pr_i ( ))_i

i n

P S S P S eys S

=

∏

HS = « present/absent » (1/0)

P(HS=1|OF=0, CF=1)

Joint probability distribution over species occurrences:

(15)

 « Conserving » a species increases its survival probability

 And the survival probability of its predators

 And thus, the species richness of the food web!

 Find the « optimal » feasible set of species to conserve, given

 A conservation budget B

 Species conservation costs C_i

 A global criterion (expectation of the number of surviving species)

Food webs, bayesian networks and

« optimal » conservation

(16)

Food webs, bayesian networks and

« optimal » conservation

0.686 1 0.1041

1 0.568

0.889 0.780

1 0.5601 0.858 0.566

0.621

0.2015

Problem: Optimal conservation is too difficult in general!

(17)

Food webs, bayesian networks and conservation heuristics

0 8 15 23 31 38 46 54

8 9 10 11 12

Expected number of species surviving

0 8 15 23 31 38 46 54

0.2 0.4 0.6 0.8 1

Percentage of budget required to manage all species Standard deviation

1

2

Budgeted Maximum Coverage Node Degree

Keystone Species Optimal

Betweeness Centrality BUP Rule

Greedy Heuristic

 Structure : Alaskan Network

 20 sets of randomly generated probability tables

 Comparison of the expectation and variance of the number of surviving species

(18)

Conclusions

 A Bayesian network and combinatorial optimization model for species conservation within food webs

 Efficient heuristics for approximating optimal conservation

 No theoretical guarantee about the heuristics’ performance

 Even more difficult in the “dynamic” case (work in progress)

(19)

Application II: Spatial weeds sampling for map reconstruction

 Problem: an accurate map of weed repartition in a crop field

 is a useful tool for studying weeds populations

 but observations are costly

 « Management » decision : where to get sample observations in order to achieve a good compromise between map accuracy and sampling cost?

Sabrina Gaba

(INRA- UMR Agroécologie) Mathieu Bonneau,

Nathalie Peyrard & Régis Sabbadin (INRA-MIAT)

Alexandre Albore

(ONERA-DCSD, INRA-MIAT) Florent Teichtel

(ONERA-DCSD)

Nathalie Peyrard & Régis Sabbadin (INRA-MIAT)

(20)

Optimal adaptive sampling strategy

MRF model

Reconstruction method

We combine MRF and MDP to model the problem of designing an adaptive strategy by optimization and we propose two heuristic solutions

Adaptive strategy

Estimated weed map

How to find the best one?

(21)

Performance of adaptive sampling heuristics

Comparison with 8 static sampling strategies

Z Reg1

Z

Reg2 Reg3

Star W1 W2

(22)

Performances of adaptive sampling heuristics

 Benchmark of 6 real weed maps

 Field of 13 by 13 quadrats, sample size = 13,5 % of total

 NWC = number of well classified quadrats

 Score = NWC(best strat) – NWC (strat)

Adaptive sampling is more efficient

Adaptive heuristics

(23)

Ongoing Work : Embedding adaptive sampling heuristics on a UAV

Mathematics and AI tools to adapt

sampling strategies to a robotic context

• ONERA/ région post-doc: Alexandre Albore (2014-2015)

• Co-advisor: F. Teichtel, ONERA

Objective:

To apply Artificial Intelligence methods for crops mapping using a UAV.

AI challenges

• Automated Planning under uncertainty: optimising the sampling strategy

Computer Vision: to identify and evaluate the collected data

(24)

Ongoing Work : Embedding adaptive sampling heuristics on a UAV

First step :

• Experiments on the MORSE simulator (LAAS-CNRS)

(25)

Conclusions

 A framework and two heuristics for adaptive sampling under cost constraints

 Adaptive sampling gives more accurate maps for the same cost

 Method also applied on a problem of fire ants sampling

 The sampled system is assumed static

(26)

Application III: Collective management of crop resistance to pathogens

 Problem : Cultivar resistance enables to avoid fungicides but can be broken down

 Management decision : Where and when should we allocate resistant crops to maintain both resistance and yield?

Benjamin Borgy (ex INRA-AGIR- MIAT) Jean-Noël Aubertot (INRA-AGIR)

Nathalie Peyrard & Régis Sabbadin (INRA-MIAT)

(27)

Crop rotation canola wheat barley

Host-pathogen interaction

+ -

avirulent

+ +

virulent

susceptible resistant

Case study: blackleg on canola

t, sowing t+1,sowing

harvest

tillage Dispersion from neighbouring wheat field

stubble

(28)

A GMDP model for the control of blackleg on canola

State variables in each field

 crop

 infection intensity

 composition of pathogen population

Actions (on canola fields)

 cultivar choice (CC)

 ploughing threshold (PT)

Transition functions:

 learned from simulations (SIPPOM )

Reward:

 sum of local gross margins

(29)

Number of wheat neighbours> 1

Number of barley neighbors> 0

Number of barley neighbors> 3

CC=2 PT=1

CC=2 PT=4

CC=1 PT=1

CC=1 PT=4

0 1 2 3 4 5 6 7 8 9 10

0 5 10 15 20 25 30

0 10 20 30 40 50 60 70 80 90 100

0 5 10 15 20 25 30

27700 27800 27900 28000 28100 28200 28300 28400 28500

A tool for strategies simulation

Reward Infection intensity Proportion of virulents

Spatial repartition of actions

(30)

Conclusions

A framework for designing collective strategies by optimization

A model for comparing and simulating management strategies

 BUT no satisfying solution to the problem of design by optimization

(31)

Ongoing Work :

Towards spatial management of weeds

How to optimise ecosystem services at the landscape scale?

• thesis of J. Radoszycki, ED Math/Info, 2012-2015

• Co-advisor: S. Gaba (UMR Agroécologie, Dijon)

+

-

Methodological tools

• Factored-Actions Factored MDPs (extension of GMDP)

• Stochastic factored policies

(32)

Bonus application:

Forest Management and risk aversion

(with S. Couture and M.-J. Cros)



Multi-stand forest management under risk of windthrow

 Forest owner with different multi-aged stand plots

 Presence of windthrow risk

 Risk-aversion of the owner -> Non-linear utility function :

A MDP framework for designing multi-stand strategies

Introduction of consumption-saving decisions in order to analyze the link between these decisions and forest management.

( )

1

( ) 1 U r r

β

=

−

(33)

Forest Management and risk aversion

03/06/2015

Sensitivity analysis to

Storm risk

Risk aversion

Number of stands

N=5

N=10

(34)

Bonus application (2):

Leader-Follower MDP

Tomorrow’s talk by Anne-France Viet (MODEC)

(35)

General conclusions

 Managing ecological networks

 Networks can be: spatial, causal, …

 Management can be: control, conservation, sampling ..

 Ecosystems and agricultural systems : management share similarities

 Common tools for all these problems

 graphical models, simulation, optimization

 Computing exactly the optimal strategy is out of reach

Current research focuses on approximate resolution

(36)

Still some challenges!

Design by optimization of strategies for managing ecological networks

 How to improve solutions of problems with large factored state and action spaces?

 We have heuristic solutions but there is still room for improvement. Planning as inference and continuous optimization are one possibility…

 Dynamics

 How to sample a system where the underlying process changes through time?

 How to manage processes over an ill-know network?

 Combine network learning and control/conservation actions optimization

 Embedded systems

 Using robots/UAV for spatial management/mapping

 …

(37)

References

Food webs management

W.J.M. Probert, E. Mc Donald-Madden, N. Peyrard and R. Sabbadin. Computational issues surrounding the dynamic optimization of management of an ecological food web. ECAI 2012 Workshop AIGM.

E McDonald-Madden, R Sabbadin, P.W.J. Baxter, I Chadès, E.T. Game and H.P. Possingham. Using food webs to manage ecosystems, under review.

Adaptive spatial sampling

N Peyrard, R Sabbadin, D Spring, B Brook and R Mac Nally. Model-based adaptive spatial sampling for occurrence map construction. Statistics and Computing, 1-14, 2011.

M. Bonneau, N. Peyrard and R. Sabbadin. A Reinforcement-Learning Algorithm for Sampling Design in Markov Random Fields, ECAI 2012

M. Bonneau, S. Gaba, N. Peyrard and R. Sabbadin. Reinforcement learning-based design of sampling policies under cost constraints in Markov random fields: Application to weed map reconstruction, CSDA, 2014

A. Albore, N. Peyrard, R. Sabbadin and F. Teichteil. An Online Replanning Approach for Crop Fields Mapping with Autonomous UAVs, ICAPS, 2015.

GMDP, FA-FMDP, applications to agro-ecological processes management

N Peyrard, R Sabbadin, E Lo-Pelzer and J.N. Aubertot. A graph-based Markov decision process applied to the optimization of strategies for integrated management of diseases. Phytopatthology, 2007.

R Sabbadin, N Peyrard and N Forsell. A framework and a mean-field algorithm for the local control of spatial processes. International Journal of Approximate Reasoning, 2011.

J. Radoszycki, N. Peyrard and R. Sabbadin. Finding good stochastic factored policies for factored Markov decision processes, ECAI (poster), 2014.

(38)

(39)

The models

 Models based on Stochastic graphical models

 Bayesian networks

 Markov Random Fields

 (Factored) Markov Decision Processes

(40)

Bayesian networks

P(HS|OF,CF)

1 1..

( ,..., _n) _i( | Pr_i ( ))_i

i n

P S S P S eys S

=

∏

Joint probability model:

Conditional probabilities

Concisely express joint probability distributions over sets of variables

A directed acyclic graph over variables

Conditional probability tables

(41)

Markov random fields

A framework for representing uncertain knowledge about spatial processes

Network representation of a spatial process

Undirected graph with cycles

A set of potential functions over cliques:

MRF probability distribution:

( )

_c

( )

_c

c C

P x x

∈

∝ ∏ Ψ

( ) 0,

c

x

c

x

c

Ψ > ∀

c

( ) x

c

Ψ

(42)

Graph-based Markov Decision Processes

 Several state variables {Xi}

Structured problems of sequential decision under uncertainty

(43)

Graph-based Markov Decision Processes

 Several state variables {Xi}i∈V and decision variables {Di}i∈V

Structured problems of sequential decision under uncertainty

(44)

Graph-based Markov Decision Processes

 Several state variables {Xi}i∈V and decision variables {Di}i∈V

A factored stochastic transitions model:

(

^t ¹ ^t^, ^t

)

ⁱ

(

ⁱ^t ¹ ^t^{N i}^{( )}^, ⁱ^t

)

i V

p x ⁺ x d p x ⁺ x d

∈

=

∏

Structured problems of sequential decision under uncertainty

(45)

Graph-based Markov Decision Processes

 Several state variables {Xi}i∈V and decision variables {Ai}i∈V

A factored stochastic transitions model

A local reward model

(

^t

^,

^t

^,

^t ¹

)

ⁱ

(

ⁱ^t

^,

ⁱ^t

^,

ⁱ^t ¹

)

i V

r x d x

⁺

r x d x

⁺

∈

= ∑

Structured problems of sequential decision under uncertainty

(46)

Graph-based Markov Decision Processes

Optimization problem:

Find a policyδ : X → A assigning an action δ(x) to every possible states of the system, maximizing the expected discounted sum of future rewards

Local policies (approximate):

Structured problems of sequential decision under uncertainty

( )

{

^δⁱ ^x^{N i}

}

_{i V}_∈