Measuring the interactions among variables of functions over[0, 1]n

Measuring the interactions among variables of functions over [ 0, 1 ]

Jean-Luc Marichal and Pierre Mathonet

University of Luxembourg

Measure of influence of variables

Problem

Letf∶ [0,1]ⁿ→R

We want to measure theinfluence (importance) of x_k over f(x₁, . . . ,x_k, . . . ,x_n)

Example: affine function

f(x1, . . . ,xn) =c0+c1x1+ ⋯ +ckxk+ ⋯ +cnxn

Measure of influence ofx_k:

coefficient c_k

Measure of influence of variables

More complicated functions...

f(x) = ∏ⁿ

i=1

x_i^ci

f(x) = (∑ⁿ

i=1

c_ix_i²)

1/2

f(x) = max

i=1..nmin(ci,xi)

etc.

Measure of influence of variables

A reasonable answer:

Ð→Approximation off by an affine function

f1(x1, . . . ,xn) =a0+a1x1+ ⋯ +a_kx_k + ⋯ +anxn

Measure of influence ofx_k overf: coefficient a_k

Measure of interaction among variables

What about interactions among variables?

Example: multilinear function

f(x1,x2) =c0+c1x1+c2x2+c12x1x2

Measure ofinteractionbetween x₁ andx₂ withinf: coefficient c₁₂

c₁₂=0 : zero interaction c12>0 : positive interaction c12<0 : negative interaction

Measure of interaction among variables

LetS ⊆N= {1, . . . ,n}

Measure of interaction among variables{x_k ∶k∈S}

Ð→Approx. of f by a multilinear polynomial of degree⩽s= ∣S∣

f_s(x) = ∑

T⊆N

∣T∣⩽s

a_s(T) ∏

i∈T

x_i

Measure of interaction among{x_k ∶k ∈S} inside f: coefficient a_s(S)

(leading coefficients)

Approximation problem

Multilinear approximation

Denote by M_s the set of all multilinear polynomials g∶ [0,1]ⁿ →R of degree⩽s

g(x) = ∑

T⊆N

∣T∣⩽s

as(T) ∏

i∈T

x_i

We define thebest s-th approximationof a function f ∈L²([0,1]ⁿ) as the multilinear polynomialf_s∈M_s that minimizes

d(f,g)²= ∫_[0,1]n(f(x) −g(x))²dx among allg ∈M_s

Interaction index

From the bests-th approximationfs of f, we define the following interaction index(power index ifs =1)

I(f,S) =a_s(S) (s = ∣S∣)

= coefficient of ∏

i∈S

x_i in the best s-th approximationf_s off

Interaction index

Explicit expression

I(f,S) =12^s∫_[0,1]nf(x) ∏

i∈S

(x_i−1 2)dx

Question: How can we see that this index actually measures an interaction?

Interaction index

Interpretation:

D_kf(x) = rate of change w.r.t.x_k of f atx

= local contributionof x_k overf atx DjDkf(x) = local interactionbetween xj and xk within f atx

Theorem

Iff is sufficiently differentiable, then

I(f,S) = ∫_[0,1]nq_S(x)D_Sf(x) dx whereqS(x)is a probability density function

Interaction index

What iff is not differentiable?

S-derivative→discrete S -derivative (S-difference quotient)

Theorem We have

I(f,S) = ∫_x∈[0,1]n∫_h

S∈[0,1−xS]

p_S(h) ∆^S_hf(x)

∏i∈Sh_i dh_S dx wherep_S(h)is a p.d.f. over the domain of integration

Interaction index

Examples:

f(x) = ∑ⁿ

i=1

w_ix_i

I(f,{k}) =wk

I(f,S) =0 ∣S∣ ⩾2

f(x) = (∏ⁿ

i=1

x_i)^1/n

I(f,S) = (_n+1ⁿ )ⁿ(_n+2⁶ )^∣S∣

Interaction index

Further examples:

f(x) =min

i∈T x_i (T ⊆N) I(f,S) =⎧⎪⎪

⎨⎪⎪⎩

6^∣S∣_{(∣S∣+∣T}^{∣S∣!∣T∣!}_∣+1)!, ifT ⊇S,

0, otherwise.

f(x) =max

i∈T xi (T ⊆N) I(f,S) =⎧⎪⎪

⎨⎪⎪⎩

(−1)^∣S∣+16^∣S∣(∣S∣+∣T∣+1)!^{∣S∣!∣T∣!} , ifT ⊇S,

0, otherwise.

Interaction index

Further examples:

f(x) = ∑

T⊆N

c(T) ∏

i∈T

ϕ_i(x_i) I(f,S) = ∑

T⊇S

c(T) ∏

i∈T∖SI(ϕ_i,∅) ∏

i∈S I(ϕ_i,{i})

f(x) = ∑

T⊆N

c(T)min

i∈T xi

I(f,S) =6^∣S∣ ∑

T⊇S

c(T)(∣S∣+∣T∣+1)!^{∣S∣!∣T∣!}

Some properties

A variablexk is inefficientfor f iff does not depend on xk

f(x_k;x_N∖k) −f(0_k;x_N∖k) =0

Property

Ifx_k is inefficientfor f then

I(f,S) =0 ∀S ∋k

Some properties

A variablex_k is dummy for f if

f(x_k;x_N∖k) −f(0_k;x_N∖k) =f(x_k;0_N∖k) −f(0_k;0_N∖k) (the marginal contribution ofx_k is independent of x_N∖k)

Property

Ifx_k is dummy for f then

I(f,S) =0 ∀S∋k, ∣S∣ ⩾2

Some properties

A combination of variables{x_k ∶k∈S}is dummy for f if f(x_S;x_N∖S) −f(0_S;x_N∖S) =f(x_S;0_N∖S) −f(0_S;0_N∖S)

Property

If{xk ∶k∈S} isdummy for f, then

I(f,T) =0 ∀T such that T∩S ≠ ∅ andT ∖S≠ ∅

Conclusion

Given a functionf ∈L²([0,1]ⁿ), we have defined an index to measure

the influence of variables over f

the interaction among variables within f Interpretations

leading coefficients in multilinear approximation (multilinear regression)

meanS-derivative orS-difference quotient

Properties

inefficient and dummy variables ...

Thank you for your attention!

arXiv : 0912.1547