en fr

HAL Id: hal-02597181

https://hal.inrae.fr/hal-02597181

Submitted on 6 Jul 2020

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Distributed under a Creative Commons Attribution - NonCommercial - NoDerivatives| 4.0 International License

Adaptive approximate Bayesian computation for complex models

Maxime Lenormand, Franck Jabot, Guillaume Deffuant

To cite this version:

Maxime Lenormand, Franck Jabot, Guillaume Deffuant. Adaptive approximate Bayesian computation for complex models. 8th World Congress in Probability and Statistics, Jul 2012, Istanbul, Turkey.

pp.11. �hal-02597181�

Adaptive approximate Bayesian computation for complex models

Maxime Lenormand, Franck Jabot & Guillaume Deffuant

Laboratory of Engineering for Complex Systems Irstea of Clermont-Ferrand

8 ^th World Congress in Probability and Statistics July  ^th 

To bet t er r ef l ect i t s mi ssi ons, Cemagr ef becomes I r st ea.

This publication has been funded under the PRIMA (Prototypical policy impacts on multifunctional activities in rural municipalities) collaborative project, EU 7th Framework Programme (ENV 2007-1), contract no. 212345.

Plan

1 Motivation

2 Approximate Bayesian Computation (ABC)

3 Approximate Bayesian Computation Sequential Monte Carlo

4 Adaptive Population Monte Carlo Approximate Bayesian Computation

5 Comparison of the algorithms

6 The SimVillages Model

7 Conclusion

Motivation

Complex social model Individual-based model Stochastic

High dimensional parameter space High computational cost by simulation

Estimate the parameter values Calibrate the model

Understand the model behaviour Uncertainty analysis

Validation

Plan

1 Motivation

2 Approximate Bayesian Computation (ABC)

3 Approximate Bayesian Computation Sequential Monte Carlo

4 Adaptive Population Monte Carlo Approximate Bayesian Computation

5 Comparison of the algorithms

6 The SimVillages Model

7 Conclusion

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

sss sss ss

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1 Sample θ ^∗ ∼ π(θ).

2 Simulate x ∼ f (x |θ ^∗ ).

3 If ρ(S(x ), S(y )) ≤ , accept θ ^∗ , otherwise reject.

4 Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

sss sss ss

(Pritchard et al., 1999) Derived from T. Toni 2011

Plan

1 Motivation

2 Approximate Bayesian Computation (ABC)

3 Approximate Bayesian Computation Sequential Monte Carlo

4 Adaptive Population Monte Carlo Approximate Bayesian Computation

5 Comparison of the algorithms

6 The SimVillages Model

7 Conclusion

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s

s

s ss

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s

s

s

s

sss

s

s

s s

ss

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s

w _i ^(t) = ^π(θ

(t)

i )

P N j=1 w (t−1)

j σ −1

t−1 ϕ

σ −1

t−1 (θ (t) i −θ (t−1)

j )

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s

w _i ^(t) = ^π(θ

(t)

i )

P N j=1 w (t−1)

j σ −1

t−1 ϕ

σ −1

t−1 (θ (t) i −θ (t−1)

j )

6 K 2

s

K t (θ ^∗ |θ) = σ _t−1 ⁻¹ ϕ

σ _t−1 ⁻¹ (θ ^∗ − θ)

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s

w _i ^(t) = ^π(θ

(t)

i )

P N j=1 w (t−1)

j σ −1

t−1 ϕ

σ −1

t−1 (θ (t) i −θ (t−1)

j )

6 K 2

s

K t (θ ^∗ |θ) = σ _t−1 ⁻¹ ϕ

σ _t−1 ⁻¹ (θ ^∗ − θ)

- ρ(x , y ) ≤ ₂

s

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s

w _i ^(t) = ^π(θ

(t)

i )

P N j=1 w (t−1)

j σ −1

t−1 ϕ

σ −1

t−1 (θ (t) i −θ (t−1)

j )

6 K 2

s

K t (θ ^∗ |θ) = σ _t−1 ⁻¹ ϕ

σ _t−1 ⁻¹ (θ ^∗ − θ)

- ρ(x , y ) ≤ ₂

ss

s

s

s

ss

ss ss

sss s

ABC SMC

Algorithm

(Beaumont et al., 2009) (Toni et al., 2009)

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s

w _i ^(t) = ^π(θ

(t)

i )

P N j=1 w (t−1)

j σ −1

t−1 ϕ

σ −1

t−1 (θ (t) i −θ (t−1)

j )

6 K 2

s

K t (θ ^∗ |θ) = σ _t−1 ⁻¹ ϕ

σ _t−1 ⁻¹ (θ ^∗ − θ)

- ρ(x , y ) ≤ ₂

ss

s s s ss ss ss

sss s sss ssss

s

sss

ABC SMC

Recent Improvements

Determination "on-line" of the decreasing sequence of tolerance levels { ₁ , ..., _T }. (DelMoral et al. (2011)) and (Drovandi and Pettitt (2011).

From a quadratic complexity to a linear complexity for the computation of the weights thanks to a MCMC kernel.

(DelMoral et al. (2011)) and (Drovandi and Pettitt (2011).

Realisation of M pseudo-observations (DelMoral et al.

(2011))

= ⇒ The MCMC kernel leads to the problem of particles

duplication!!!

Plan

1 Motivation

2 Approximate Bayesian Computation (ABC)

3 Approximate Bayesian Computation Sequential Monte Carlo

4 Adaptive Population Monte Carlo Approximate Bayesian Computation

5 Comparison of the algorithms

6 The SimVillages Model

7 Conclusion

Adaptive PMC ABC

Algorithm

Prior

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾

6

? K ₁

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾

6

? K _{1 s} ^s

s sss s N − N _α particles

p acc = P N

k=N α +1 1 ρ(x ,y)≤ ₁

N − N _α

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k=N α +1 1 ρ(x,y)≤ ₁

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k=N α +1 1 ρ(x,y)≤ ₁

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

-

N α

₂

s ss s s s ss ss s s

(Lenormand et al.)

Adaptive PMC ABC

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k=N α +1 1 ρ(x,y)≤ ₁

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

-

N α

₂

s ss s s s ss ss s s

· · · _{{p acc <p}

accmin }

Posterior sss sss sss

(Lenormand et al.)

Adaptive PMC ABC

Pros and Cons

Pros

Control the number of simulations at each iteration (N − N _α simulations).

Determination "on-line" of the decreasing sequence of tolerance levels.

No problem of particles duplication. Stopping criterion

Cons

Complexity O(N α 2 ) for the computation of the weights

Adaptive PMC ABC

Pros and Cons

Pros

Control the number of simulations at each iteration (N − N _α simulations).

Determination "on-line" of the decreasing sequence of tolerance levels.

No problem of particles duplication. Stopping criterion

Cons

Complexity O(N α 2 ) for the computation of the weights

Adaptive PMC ABC

Pros and Cons

Pros

Control the number of simulations at each iteration (N − N _α simulations).

Determination "on-line" of the decreasing sequence of tolerance levels.

No problem of particles duplication.

Stopping criterion Cons

Complexity O(N α 2 ) for the computation of the weights

Adaptive PMC ABC

Pros and Cons

Pros

Control the number of simulations at each iteration (N − N _α simulations).

Determination "on-line" of the decreasing sequence of tolerance levels.

No problem of particles duplication.

Stopping criterion

Cons

Complexity O(N α 2 ) for the computation of the weights

Adaptive PMC ABC

Pros and Cons

Pros

Control the number of simulations at each iteration (N − N _α simulations).

Determination "on-line" of the decreasing sequence of tolerance levels.

No problem of particles duplication.

Stopping criterion Cons

Complexity O(N α 2 ) for the computation of the weights

Plan

1 Motivation

2 Approximate Bayesian Computation (ABC)

3 Approximate Bayesian Computation Sequential Monte Carlo

4 Adaptive Population Monte Carlo Approximate Bayesian Computation

5 Comparison of the algorithms

6 The SimVillages Model

7 Conclusion

Comparison of the algorithms

Toy Example : Presentation

f(x|θ) ∼ 1 2 φ

θ, 1

100

+ 1

2 φ (θ, 1) and θ ∼ U _[−10,10]

Probability

−3 −2 −1 0 1 2 3

0.0 0.5 1.0 1.5 2.0 2.5

θ