en fr it

HAL Id: hal-02596122

https://hal.inrae.fr/hal-02596122

Submitted on 6 Jul 2020

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Calibrating a complex social model

Maxime Lenormand, Guillaume Deffuant, S. Huet

To cite this version:

Maxime Lenormand, Guillaume Deffuant, S. Huet. Calibrating a complex social model. European

Conference on Complex Systems ECCS’11, Sep 2011, Vienne, Austria. �hal-02596122�

Calibrating a complex social model

Maxime Lenormand, Guillaume Deffuant & Sylvie Huet

Laboratory of Engineering for Complex Systems Cemagref of Clermont-Ferrand

ECCS’11 September 



This publication has been funded under the PRIMA (Prototypical policy impacts on multifunctional activities in rural municipalities) collaborative project, EU 7th Framework Programme (ENV 2007-1), contract no. 212345.

Motivation

Complex social model Individual-based model Stochastic

High dimensional parameter space High computational cost by simulation

Estimate the parameter values Calibrate the model

Understand the model behaviour Uncertainty analysis

Validation

Summary

1

Approximate Bayesian Computation (ABC)

2

Adaptive approximate Bayesian computation for complex models

3

The PRIMA model

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target s

s s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

sss sss ss

(Pritchard et al., 1999) Derived from T. Toni 2011

Approximate Bayesian Computation

1

Sample θ ^∗ ∼ π(θ).

2

Simulate x ∼ f (x |θ ^∗ ).

3

If ρ(x , y ) ≤ , accept θ ^∗ , otherwise reject.

4

Repeat until a sample of the desired size is obtained

Prior distribution π(θ)

Posterior distribution π(θ)P θ {f (x |θ) = y }

6

- Simulation

b

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

&%

'$

Target

s s

s s s s s

s s s

s s

s

sss sss ss

(Pritchard et al., 1999) Derived from T. Toni 2011

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s

s

s ss

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s

s

s

s

sss

s

s

s s

ss

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

-

w _i ⁽¹⁾ s

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s 6 K ₂

s

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s 6 K ₂

s -

ρ(x , y ) ≤ ₂ s

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s 6 K ₂

s -

ρ(x , y ) ≤ ₂ ss

s

s

s

ss

ss ss

sss s

ABC SMC (Sequential Monte-Carlo)

Algorithm

(Sisson et al., 2007)

(Beaumont et al., 2009) Derived from T. Toni 2011

Prior Posterior

_T

_{1 2}

· · ·

s s s s

s s s

s s

s s

s s s s ss

s s s s sss s s s s ss

- w _i ⁽¹⁾ s 6 K ₂

s -

ρ(x , y ) ≤ ₂ ss

s s s ss ss ss

sss s sss ssss

s

sss

ABC SMC (Sequential Monte-Carlo)

Issues related to the model complexity

How to control the number of simulations?

How to determine the sequence of tolerance levels { ₁ , ..., T }?

When to stop the algorithm?

Summary

1

Approximate Bayesian Computation (ABC)

2

Adaptive approximate Bayesian computation for complex models

3

The PRIMA model

Adaptive ABC SMC for complex models

Algorithm

Prior

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾ ₆

? K ₁

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

- w _i ⁽¹⁾ ₆

? K _{1 s} ^s

s sss s N − N _α particles

p acc = P N

k=N

+1 1 ρ(x,y)≤

N − N _α

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k =N

+1 1 ρ(x,y)≤

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k =N

+1 1 ρ(x,y)≤

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

-

N α

₂

s ss s s s ss ss s s

(Lenormand et al.)

Adaptive ABC SMC for complex models

Algorithm

Prior N particles

(LHS)

s s s s

s s s

s s

s s

s s s s ss

-

N α

₁

s s s s

s s

s s

s s s

s s s sss s N − N _α particles

p acc = P N

k =N

+1 1 ρ(x,y)≤

N − N _α s s s sss s

s s s s

s s

s s

s s s N particles

-

N α

₂

s ss s s s ss ss s s

· · · _{{p acc <p}

accmin }

Posterior sss sss sss

(Lenormand et al.)

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

= ⇒ N − N _α simulations at each iteration

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

= ⇒ N − N _α simulations at each iteration

How to determine the sequence of tolerance levels

{ ₁ , ..., T }?

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

= ⇒ N − N _α simulations at each iteration

How to determine the sequence of tolerance levels { ₁ , ..., T }?

= ⇒ t = α-quantile of the N distances to the data

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

= ⇒ N − N _α simulations at each iteration

How to determine the sequence of tolerance levels { ₁ , ..., T }?

= ⇒ _t = α-quantile of the N distances to the data

When to stop the algorithm?

Adaptive ABC SMC for complex models

Issues related to the model complexity

How to control the number of simulations?

= ⇒ N − N _α simulations at each iteration

How to determine the sequence of tolerance levels { ₁ , ..., _T }?

= ⇒ t = α-quantile of the N distances to the data When to stop the algorithm?

= ⇒ p acc < p acc

Adaptive ABC SMC for complex models

Toy example: Presentation

f (x |θ) ∼ 1 2 φ

θ, 1

100

+ 1

2 φ (θ, 1) and θ ∼ U _[−10,10]

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

Probability

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

θ

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

Probability

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

θ

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

Probability

−3 −2 −1 0 1 2 3

0.00.51.01.52.02.5

θ

Adaptive ABC SMC for complex models

Toy example: Parameters Study

N _α = 5000; α from 0.9 to 0.1 corresponding to N = 5555 to 50000

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations L

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations

L

Adaptive ABC SMC for complex models

Toy example: Parameters Study

N _α = 5000; α from 0.9 to 0.1 corresponding to N = 5555 to 50000

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations L

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations

L

Adaptive ABC SMC for complex models

Toy example: Parameters Study

N _α = 5000; α from 0.9 to 0.1 corresponding to N = 5555 to 50000

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations L

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations

L

Adaptive ABC SMC for complex models

Toy example: Parameters Study

N _α = 5000; α from 0.9 to 0.1 corresponding to N = 5555 to 50000

●

●

●

● ●

● ●

● ●● ● ● ● ● ●●●

● ● ● ● ● ● ●●●

●

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations L

1e+05 2e+05 3e+05 4e+05 5e+05

0.02 0.04 0.06 0.08 0.10 0.12 0.14

Number of simulations

L

Adaptive ABC SMC for complex models

Toy example: Model comparison

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

0e+00 2e+05 4e+05 6e+05 8e+05 1e+06

0.000.050.100.150.20

Number of simulations L2

Summary

1

Approximate Bayesian Computation (ABC)

2

Adaptive approximate Bayesian computation for complex models

3

The PRIMA model

The PRIMA model

Parameters and summary statistics

4 parameters

8 summary statistics k(ρ _m (S m , S

_m )) _1≤m≤M k

∞ = sup _1≤m≤M |ρ _m (S m , S

_m )|

The PRIMA model

Acceptance rate and threshold evolution

0 20 40 60 80 100 120

0.00.10.20.30.40.50.6

Number of iterations

Acceptance rate

0 20 40 60 80 100 120

−1.5−1.0−0.50.00.51.0

Number of iterations

Tolerance threshold

The PRIMA model

Posterior density of a parameter

1.0 1.5 2.0 2.5 3.0 3.5

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

θ

Density

The PRIMA model

Concrete results

1.4 second by simulations 400,000 simulations 6 days

Age pyramid

050100150200

Conclusion

We have answered the three research questions Comparison with a well-known method

Calibration of a complex social model