(Not that) Old and New Results on Probabilistic Intersection Type Systems

(Not that) Old and New Results on

Probabilistic Intersection Type Systems

Flavien BREUVART, Ugo Dal Lago

Focus, Inria team, Bologna

GdR-LL: June 12-13 2016

Randomized Programming

Motivations

Randomized Algorithmics Efficiency analysis

Cryptography Security Machine Learning

Modeling

Probabilistic

-calculus (Weak head-reduction)

M,N :₌ x_|λx.M_|M N_|M_⊕N (_λx.M)N ¹ M[N/x]

M_⊕N

M N

1 2

1 2

YA0 ^∗_• I

YA1 ^∗_•

• I

YA2 ^∗_• ^•

• I

YA3

A:_=λux.x(_λy.(u(Sx)_⊕y)I Prob(YA0_⇓)_<1

1 1 2

1 2

1 1 2

1 2

1 2 1 2

1 1 2

1 2

1 1 2 2

1 2

Probability of Normalization:

A difficult problem

Problem PConv:

What is the probability pof convergence of a given programM?

PConv is

-complete:

one derivation is insufficient

The following(and any variation) isimpossibleto getfrom r.e. type systems:

Prob(M_⇓)₌p _⇔ ^π

`M:p_·α

Probabilistic dependency is not type dependency

Probabilistic Dependency non-uniform; randomized;

only propagate dependency Type Dependency uniform; non-deterministic;

waiting for data to propagate

Some Recursion Theory

The lower-bound problem is

-complete

Asking whether Prob(M_⇓)_>p is recursively enumerable.

The upper-bound problem is

-complete

Asking whether Prob(M_⇓)_<p is a problem of the form:

∃x_∀y,φ(x,y)

The exact bound problem (PConv) is

-complete

Asking whether Prob(M_⇓)₌p is a problem of the form:

∀x_∃y,φ(x,y)

Non deterministic IT systems:

Idempotent Intersection Types

Grammar

(types) _α,β :_{= ∗ |}a_→α (intersections) a,b finite sets of types

(weight) p,q _∈{0, ...,1}

(context) _Γ,∆ :₌ _ _| x:a,Γ (sequent) s :_{= Γ`}M:_α

Weak head reduction Rule furnishing a type to_λx.Ω

Γ,x:{

[

α

]

}`x:

1_·

α

Γ`M:

p_·

α Γ`M_⊕N:

p 2·

α

Γ`N:<sub>α</sub> Γ`M_⊕N:

p 2·

α Γ`λx.M:

1_·

∗ Γ;x:a_`M:

p_·

α Γ`λx.M:

1_·(

a_→

p_·

α

)

Γ`M:

p_·(

a_→

q_·

β

)

nΓ

h∆α

`N:

r_α·

α¯

¯

¯α∈a

i

o

Γ

+P∆α

`M N:

pq(^Q_ir_i)_·

β

Non-deterministic IT systems:

Non-idempotent IT [De Carvalho]

Grammar

(types) _α,β :_{= ∗ |}a_→α

(intersections) a,b finite multisets of types

(weight) p,q _∈ {0, ...,1}

(context) _Γ,∆ :₌ _ _| x:a,Γ (sequent) s :_{= Γ`}M:_α

Multisets of intersections Multiplicities corresponding to

different uses

Γ,

x: [_α]_`x:

1_·

α

Γ`M:

p_·

α Γ`M_⊕N:

p 2·

α

Γ`N:<sub>α</sub> Γ`M_⊕N:

p 2·

α

Γ

`λx.M:

1_·

∗ Γ;x:a_`M:

p_·

α Γ`λx.M:

1_·(

a_→

p_·

α

)

Γ`M:

p_·(

a_→

q_·

β

)

h∆α`N:

r_α·

α¯

¯

¯α∈aⁱ Γ+P∆α`M N:

pq(^Q_ir_i)_·

β

Probabilistic IT systems:

Weight monad [Ehrhard et al]

Grammar

(types) _α,β :_{= ∗ |}a_→p_·α

(intersections) a,b finite multisets of types (weight) p,q _∈ {0, ...,1}

(context) _Γ,∆ :₌ _ _| x:a,Γ (sequent) s :_{= Γ`}M:p_·α

Probability for the function to get

this result Sums weighted but non-deterministic

Γ,

x: [_α]_`x:1_·α

Γ`M:p<sub>·α</sub> Γ`M_⊕N:^p_2·α

Γ`N:<sub>α</sub> Γ`M_⊕N:^p_2·α

Γ

`λx.M:1_·∗

Γ;x:a<sub>`</sub>M:p<sub>·α</sub> Γ`λx.M:1_·(a_→p_·α)

Γ`M:p<sub>·</sub>(a<sub>→</sub>q<sub>·β</sub>) <sup>h</sup><sub>∆α`</sub>N:r_α·α^¯¯_¯α∈aⁱ Γ+P∆α`M N:pq(^Q_ir_i)_·β

Probabilistic IT systems:

Weight monad [Ehrhard et al]

Grammar

(types) _α,β :_{= ∗ |}a_→p_·α

(intersections) a,b finite multisets of types (weight) p,q _∈ {0, ...,1}

(context) _Γ,∆ :₌ _ _| x:a,Γ (sequent) s :_{= Γ`}M:p_·α

A derivation is an execution the weight is the probability to get this

execution

M: ¹

2³·α •

•

•

•

•

•

•

λx.N:1_·α

•

1

1 2

1 2

1 2

1 2

1

1 2

1 2

Summing all possible executions?

The (naive) theorem

Prob(M_⇓) ₌ ^Xhp^¯¯_¯ ^π

`M:p_·α i

Not summing too much

Issue:

`λx.x:1_·∗

x: [_α]_`x:1_·α

`λx.x:1_·([_α]_→1_·α)

Solution:

Ph

p^¯¯_¯ ^π

`M:p_·∗

i

Summing all of them

Issue:

M: [_α,α]_→∗

N₁:1_·α N_1⊕N₂:¹_2α

N₂:1_·α N_1⊕N₂:¹_2α M (N1⊕N2) :¹_4·∗

Solution:

Use keyed-multisets

Theorem and implications

The theorem

Prob(M_⇓) ₌ ^XW(M) whereW(M)₌^hp^¯¯_¯ ^π

`M:p_·∗

i

Lower-bound problem (

)

Prob. of conv. ofM _>q iff

∃P_⊆f W(M), X p∈P

p_>q

Upper-bound problem (

)

Prob. of conv. ofM _<q iff

∃q⁰,∀P_⊆fW(M), X p∈P

p _<q⁰ _< q

The exact bound problem is

Sketching the proof

Cut Elimination

π

`M:p_·∗

π⁰

`M⁰:^p_q·∗

such that:

•

•

Small-step distrib.

ΛR ΠR

Λ Π

deriv.

deriv.

red red

Value determinism

∀normal formV, Unicity of derivation

`V:1_·∗

`λx.M:1_·∗

Big-step distribution

Λ Π

Λv ΠV

deriv.

deriv.

eval eval

Conclusion

X

V

P(M_→V)₌^XW(M)

Toward distributions of derivations

Unsatisfactory completeness

Uninformative derivations Each derivation carries the

information regarding onepossible execution.

Quantifying over ALL derivations Distinguishing derivations and looking for new one is not really

computationally-friendly.

A monad of probabilistic distributions over types

(types) _α,β :_{= ∗ |}a_→τ

(intersections) a,b finite multisets of types (monadic distributions) _ρ,τ finite distribution over types

A comonad of probabilistic distributions over intersections

(types) _α,β :_{= ∗ |}A_→p_·α

(intersections) a,b finite multisets of types

(comonadic distributions) A,B finite distribution over intersections

Toward distributions of derivations

Unsatisfactory completeness

Uninformative derivations Each derivation carries the

information regarding onepossible execution.

Quantifying over ALL derivations Distinguishing derivations and looking for new one is not really

computationally-friendly.

A monad of probabilistic distributions over types

(types) _α,β :_{= ∗ |}a_→τ

(intersections) a,b finite multisets of types (monadic distributions) _ρ,τ finite distribution over types

A comonad of probabilistic distributions over intersections

(types) _α,β :_{= ∗ |}A_→p_·α

(intersections) a,b finite multisets of types

(comonadic distributions) A,B finite distribution over intersections

An unavoidable issue:

Probabilistic dependency

x : 1 2 []

1

2 [

]

c

x : 1 2

1 2

`λ

x

c

x :

1 2 []

1

2 [

]

1 2

1 2

o

`

c

: 1 2

`

(

x

c

x ) (c

) : 1 2

1 4

No such a rule

No reasonable and systematic rule can perform this step.

Forgotten dependency

An unavoidable issue:

Probabilistic dependency

x : 1 2 []

1

2 [

]

c

x : 1 2

1 2

`λ

x

c

x :

1 2 []

1

2 [

]

1 2

1 2

o

`

c

: 1 2

`

(

x

c

x ) (c

) : 1 2

1 4

No such a rule

No reasonable and systematic rule can perform this step.

Forgotten dependency

An unavoidable issue:

Probabilistic dependency

x : 1 2 []

1

2 [

]

c

x : 1 2

1 2

`λ

x

c

x :

1 2 []

1

2 [

]

1 2

1 2

o

`

c

: 1 2

`

(

x

c

x ) (c

) : 1 2

1 4

No such a rule

No reasonable and systematic rule can perform this step.

Forgotten dependency

An unavoidable issue:

Probabilistic dependency

x : 1 2 []

1

2 [

]

c

x : 1 2

1 2

`λ

x

c

x :

1 2 []

1

2 [

]

1 2

1 2

o

`

c

: 1 2

`

(

x

c

x ) (c

) : 1 2

1 4

No such a rule

No reasonable and systematic rule can perform this step.

Forgotten dependency

Distributions over sequents

Grammar

(types) _α,β :_{= ∗ |}a_→α

(intersections) a,b finite multisets of types (context) _Γ,∆ :₌ _ _| x:a,Γ

(logical sequent) l :_{= Γ`α}

(sequent distributions) L Distributions over logical sequents (type sequent) s :₌ (_~x,M)D

(x,x)^{[_α]_`α}

MD NE

M_⊕N¹₂D₊¹₂E M^{}

M^{p_i·(_Γi;x:a_i`αi)_|i_≤n}

λx.M^{p_i·Γi`a<sub>i</sub>→αi|i<sub>≤</sub>n}+{(1<sub>−</sub><sup>P</sup><sub>i</sub>p<sub>i</sub>)<sub>· ∗</sub>} M<sup>©</sup>p<sub>·</sub>(<sub>Γ`</sub>[_αi,1, ...,αi,m_i]_→β)^¯¯i_≤n^ª _∀i,j, ND_i,j M N^Pi_≤n

n(p_i^Q_j≤miD_i,j(_{∆i,j`αi,j</sub>))<sub>·</sub>(<sub>Γ+</sub><sup>P</sup><sub>j∆i,j`}:_βi)^¯¯∀(_∆ij)_j^o

Distributions over sequents

Grammar

(types) _α,β :_{= ∗ |}a_→α

(intersections) a,b finite multisets of types (context) _Γ,∆ :₌ _ _| x:a,Γ

(logical sequent) l :_{= Γ`α}

(sequent distributions) L Distributions over logical sequents (type sequent) s :₌ (_~x,M)D

Theorem

Prob(M_⇓) ₌ SupⁿD(_{` ∗})^¯¯_¯ ^π MD

o

Conclusion

Past

[Ehr,Pag,Tas]

A fully abstract model for Λ⊕.

Sketched intersection type

system.

Now

Refinment of the ITS.

Combinatorial and simpler proof of

completeness.

Collapsed intersection type system.

future

Fixpoint/coninductive proofs

unbounded terms:

while coin continue Σ⁰₁, cherckable or

inferenceable fragments