TYPE INFERENCE

(1)

François Pottier

The Programming Languages Mentoring Workshop @ ICFP August 30, 2015

(2)

What is type inference ?

What is the type of this OCaml function ?

let f verbose msg = if verbose then msg else ""

Type inference is mostly a matter offinding out the obvious.

(3)

What is type inference ?

OCaml infers it :

# let f verbose msg = if verbose then msg else "";;

val f : bool -> string -> string = <fun>

(4)

OCaml infers it :

# let f verbose msg = if verbose then msg else "";;

val f : bool -> string -> string = <fun>

Type inference is mostly a matter offinding out the obvious.

(5)

Where is type inference ?

Everywhere.

I C++ and Java have a bit more

I C++ hasauto,decltype, inference of template parameters...

I Java infers type parameters to method calls andnew(slowly... see next)

I Scala has a lot

I a form of “local type inference”

I “bidirectional” (bottom-up in places, top-down in others)

I SML, OCaml, Haskell have a lot, too

I “non-local” (based on unification /constraint solving)

I Haskell, Scala, Coq, Agda infer not just types, but also terms (that is, code) !

(6)

Everywhere.

Everytyped programming language hassometype inference.

I Pascal, C, etc. have a tiny amount

I the type of every expression is “inferred” bottom-up

I C++ and Java have a bit more

I C++ hasauto,decltype, inference of template parameters...

I Java infers type parameters to method calls andnew(slowly... see next)

I Scala has a lot

I a form of “local type inference”

I “bidirectional” (bottom-up in places, top-down in others)

I SML, OCaml, Haskell have a lot, too

I “non-local” (based on unification /constraint solving)

I Haskell, Scala, Coq, Agda infer not just types, but also terms (that is, code) !

(7)

Anyone who has ever used the“diamond”in Java 7...

List<Integer> xs = new Cons<> (1, new Cons<> (1, new Cons<> (2, new Cons<> (3, new Cons<> (3, new Cons<> (5, new Cons<> (6, new Cons<> (6, new Cons<> (8, new Cons<> (9, new Cons<> (9, new Cons<> (9, new Nil<> ()

)))))))))))); // Tested with javac 1.8.0_05

(8)

Anyone who has ever used the“diamond”in Java 7...

List<Integer> xs =

new Cons<> (1, // 0.5 seconds new Cons<> (1, // 0.5 seconds new Cons<> (2, // 0.5 seconds new Cons<> (3, // 0.6 seconds new Cons<> (3, // 0.7 seconds new Cons<> (5, // 0.9 seconds new Cons<> (6, // 1.4 seconds new Cons<> (6, // 6.0 seconds new Cons<> (8, // 6.5 seconds new Cons<> (9, // 10.5 seconds new Cons<> (9, // 26 seconds new Cons<> (9, // 76 seconds new Nil<> ()

)))))))))))); // Tested with javac 1.8.0_05

... may be interested to hear that this feature seems to have exponential cost.

(9)

How does it work ?

Should I do research in type inference ?

(10)

Benefits

What does type inference do for us, programmers ? Obviously,

I it reducesverbosityandredundancy,

I giving usstatic type checkingat little syntactic cost.

(11)

What does type inference do for us, programmers ? Obviously,

I it reducesverbosityandredundancy,

I giving usstatic type checkingat little syntactic cost.

Less obviously,

I it sometimeshelps us figure outwhat we are doing...

(12)

Example : sorting

What is the type ofsort?

let rec sort (xs : ’a list) = if xs = [] then

[]

else

let pivot = List.hd xs in

let xs1, xs2 = List.partition (fun x -> x <= pivot) xs in sort xs1 @ sort xs2

This functionnever returns a non-empty list.

(13)

What is the type ofsort?

let rec sort (xs : ’a list) = if xs = [] then

[]

else

let pivot = List.hd xs in

let xs1, xs2 = List.partition (fun x -> x <= pivot) xs in sort xs1 @ sort xs2

Oops... This is a lotmore generalthan I thought ! ? val sort : ’a list -> ’b list

This functionnever returns a non-empty list.

(14)

Example : searching a binary search tree

type ’a tree = Empty | Node of ’a tree * ’a * ’a tree What is the type offind?

let rec find compare x = function

| Empty -> raise Not_found

| Node(l, v, r) ->

let c = compare x v in if c = 0 then v

else find compare x (if c < 0 then l else r)

Good – this allows us to implement lookup in a map usingfind.

(15)

type ’a tree = Empty | Node of ’a tree * ’a * ’a tree What is the type offind?

let rec find compare x = function

| Empty -> raise Not_found

| Node(l, v, r) ->

let c = compare x v in if c = 0 then v

else find compare x (if c < 0 then l else r) It may well bemore generalthan you expected :

val find : (’a -> ’b -> int) -> ’a -> ’b tree -> ’b Good – this allows us to implement lookup in a map usingfind.

(16)

This1989 paperby Danvy and Filinski...

(17)

This1989 papercontains typing rules like this :

(18)

and this :

(19)

and this :

How does onemake senseof these rules ? How does oneguessthem ?

(20)

Well, thesemanticsofshiftandresetis known...

let return x k = k x

let bind c f k = c (fun x -> f x k) let reset c = return (c (fun x -> x))

let shift f k = f (fun v -> return (k v)) (fun x -> x) ...so their types can beinferred.

(21)

Let us introduce a little notation :

type (’alpha, ’tau, ’beta) komputation =

(’tau -> ’alpha) -> ’beta

type (’sigma, ’alpha, ’tau, ’beta) funktion =

’sigma -> (’alpha, ’tau, ’beta) komputation

(22)

Example : groking delimited continuations

What should be the typing rule forreset? Ask OCaml :

# (reset : (_, _, _) komputation -> (_, _, _) komputation);;

- : (’a, ’a, ’b) komputation -> (’c, ’b, ’c) komputation

(’aisσ,’bisτ,’cisα.)

(23)

What should be the typing rule forreset? Ask OCaml :

# (reset : (_, _, _) komputation -> (_, _, _) komputation);;

- : (’a, ’a, ’b) komputation -> (’c, ’b, ’c) komputation So Danvy and Filinski were right :

(’aisσ,’bisτ,’cisα.)

(24)

Example : groking delimited continuations

What should be the typing rule forshift? Ask OCaml :

# (shift : ((_, _, _, _) funktion -> (_, _, _) komputation) ->

(_, _, _) komputation);;

- : ((’a, ’b, ’c, ’b) funktion -> (’d, ’d, ’e) komputation) ->

(’c, ’a, ’e) komputation

(’aisτ,’bisδ,’cisα,’disσ,’eisβ.)

(25)

What should be the typing rule forshift? Ask OCaml :

# (shift : ((_, _, _, _) funktion -> (_, _, _) komputation) ->

(_, _, _) komputation);;

- : ((’a, ’b, ’c, ’b) funktion -> (’d, ’d, ’e) komputation) ->

(’c, ’a, ’e) komputation So Danvy and Filinski were right :

(’aisτ,’bisδ,’cisα,’disσ,’eisβ.)

(26)

Sometimes, type inferencehelps us figure outwhat we are doing.

(27)

In what ways could type inference be a bad thing ?

I Liberally quotingReynolds (1985), type inference allows us to make code succinct to the point of unintelligibility.

I Reduced redundancy makes it harder for the machine tolocateandexplain type errors.

Both issues can be mitigated by adding well-chosentype annotations.

(28)

How does it work ?

(29)

Let us focus on asimply-typedprogramming language.

I base types (int,bool, ...), function types (int -> bool, ...), pair types, etc.

I no polymorphism, no subtyping, no nuthin’

Type inference in this setting isparticularly simple and powerful.

(30)

Simple type inference, very informally

msghas typeα2.

The “if” expression must have typeβ. Soα1=bool. Andα2=string=β.

Solving these equations reveals thatfhas typebool -> string -> string.

(31)

Simple type inference, very informally

Sayfhas unknown typeα.

(32)

Simple type inference, very informally

fis a function of two arguments.

(33)

Simple type inference, very informally

fis a function of two arguments. Soα=α1→α2→β.

Andα2=string=β.

(34)

Simple type inference, very informally

fis a function of two arguments. Soα=α1→α2→β. verbosehas typeα1.

(35)

Simple type inference, very informally

msghas typeα2.

(36)

Simple type inference, very informally

msghas typeα2.

The “if” expression must have typeβ.

(37)

Simple type inference, very informally

msghas typeα2.

The “if” expression must have typeβ. Soα1=bool.

(38)

Simple type inference, very informally

msghas typeα2.

(39)

msghas typeα2.

(40)

Let us see how it has been explained / formalized through history...

(41)

The 1970s

(42)

Milner (1978)inventstype inferenceand MLpolymorphism.

He re-discovers, extends, and popularizes an earlier result byHindley (1969).

(43)

Milner’s description

Milner publishes a “declarative”

presentation,Algorithm W,

Algorithm J maintains acurrent substitutionin a global variable. Both composesubstitutions produced byunification, and create

“new”variables as needed.

Milner does not describe UNIFY. Naive unification (Robinson, 1965) hasexponential complexitydue to lack of sharing.

(44)

Milner’s description

presentation,Algorithm W,

Algorithm J maintains acurrent substitutionin a global variable. Both composesubstitutions produced byunification, and create

Milner does not describe UNIFY. Naive unification (Robinson, 1965) hasexponential complexitydue to lack of sharing.

(45)

Milner’s description

presentation,Algorithm W, and an “imperative” one, Algorithm J.

Both composesubstitutions produced byunification, and create

Naive unification (Robinson, 1965) hasexponential complexitydue to lack of sharing.

(46)

Milner’s description

(47)

Milner’s description

Algorithm J maintains acurrent substitutionin a global variable.

produced byunification, and create

hasexponential complexitydue to lack of sharing.

(48)

Milner’s description

hasexponential complexitydue to lack of sharing.

(49)

Milner does not describe UNIFY.

(50)

The 1980s

This leads to ahigher-level,more modularpresentation, which matches the informal explanation.

(51)

Cardelli (1987),Wand (1987)and others formulate type inference as a two-stage process :generatingandsolvinga conjunction of equations.

This leads to ahigher-level,more modularpresentation, which matches the informal explanation.

(52)

The 1990s

I They explain sharing by usingvariablesas memory addresses.

I They explain “new” variables asexistential quantification.

(53)

Kirchner & Jouannaud (1990),Rémy (1992)and others push this approach further.

I They explain constraint solving asrewriting.

I They explain sharing by usingvariablesas memory addresses.

I They explain “new” variables asexistential quantification.

(54)

Anintermediate languagefor describing type inference problems.

τ ::=α|τ→τ|. . .

C ::=⊥ |τ=τ|C∧C| ∃α.C

Aconstraint generatortransforms the program into a constraint.

Aconstraint solverdetermines whether the constraint is satisfiable (and computes a description of its solutions).

(55)

Afunctionof a type environmentΓ, a termt, and a typeτto a constraint.

Defined by cases :

~Γ`x:τ = (Γ(x) =τ)

~Γ`λx.u:τ = ∃α1α2. τ=α1→α2∧

~Γ[x7→α1]`u:α2

!

~Γ`t1t2:τ = ∃α.(~Γ`t1:α→τ∧~Γ`t2:α)

(56)

Transform the constraint, step by step, obeying a set of rewriting rules.

If :

I every rewriting steppreserves the meaningof the constraint,

I every sequence of rewriting stepsterminates,

I a constraint that cannot be further rewritten either is⊥or issatisfiable, then we have asolver, i.e., an algorithm for decidingsatisfiability.

(57)

A new variableαcan be introduced to stand for a sub-termτ:

Think ofαas theaddressofτin the machine.

Instead of duplicating a whole sub-term, one duplicates its address :

This accounts forsharing. Robinson’s exponential blowup is avoided.

(58)

Rémy works withmulti-equations, equations with more than two members :

In the machine, one maintainsequivalence classesof variables using aunion-find data structure.

The occurs-check (which detects cyclic equations) takes place once at the end.

(Doing it at every step, like Robinson, would cause aquadratic slowdown.) This is Huet’squasi-linear-timeunification algorithm (1976).

(59)

How does it work ?

(60)

Just as in a compiler, anintermediate languageis a useful abstraction.

I thinkdeclarative, not imperative

I saywhatyou want computed, not how to compute it

I build a constraint, then“optimize”it step by step until it is solved The constraint-based approach scales up and handles

I Hindley-Milner polymorphism (Pottier and Rémy, 2005)

I elaboration (Pottier, 2014)

I type classes, OCaml objects, and more.

(61)

Is type inference a hot topic ?

Not really.Not at this moment.

I A Unification Algorithm for Coq Featuring Universe Polymorphism and Overloading(Ziliani, Sozeau) I Practical SMT-Based Type Error Localization(Pavlinovic, King, Wies)

Yet, people still getdrawninto itby necessity.

I remember,everytyped programming language needssometype inference !

(62)

Is type inference a hot topic ?

At ICFP 2015, 4 out of 35 papers seem directly or indirectly concerned with it :

I 1ML – Core and Modules United (F-ing First-Class Modules)(Rossberg) I Bounded Refinement Types(Vazou, Bakst, Jhala)

(63)

At ICFP 2015, 4 out of 35 papers seem directly or indirectly concerned with it :

I 1ML – Core and Modules United (F-ing First-Class Modules)(Rossberg) I Bounded Refinement Types(Vazou, Bakst, Jhala)

Yet, people still getdrawninto itby necessity.

I remember,everytyped programming language needssometype inference !

(64)

Inference forpowerful / complextype systems.

I universal and existential types

I dependent types (Ziliani and Sozeau) and refinement types (Vazou et al.)

I linear and affine types

I subtyping

I first-class modules (Rossberg) Inference fortricky / uglylanguages.

I e.g., JavaScript – which was not designed as a typed language, to begin with Locatingandexplainingtype errors.

I show all locations, or a most likely one ? (Pavlinovic et al.) Identifyingre-usable building blocksfor type inference algorithms.

(65)

Type inferencemakes the differencebetween an awful language and a great one.

I if you care about language design, you will care about type inference

(66)

Type inference is (often) anundecidableproblem.

Type error explanation is (often) anill-specifiedproblem.

Your algorithm may “work well in practice”,

I but it could be difficult toformally arguethat it does,

I hence difficult topublish.

(67)

TYPE INFERENCE

What is type inference ?

What is type inference ?

Where is type inference ?

Benefits

Example : sorting

Example : searching a binary search tree

Example : groking delimited continuations

Example : groking delimited continuations

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

Simple type inference, very informally

The 1970s

Milner’s description

Milner’s description

Milner’s description

Milner’s description

Milner’s description

Milner’s description

The 1980s

The 1990s

Is type inference a hot topic ?

Is type inference a hot topic ?

GOOD LUCK and HAVE FUN !