Measuring the interactions among variables of functions over [0, 1]n

Measuring the interactions among variables of functions over [0, 1]

Jean-Luc Marichal and Pierre Mathonet

University of Luxembourg

Measure of influence of variables

Problem

Letf: [0,1]ⁿ →R

We want to measure theinfluence (importance) of x_k over y =f(x1, . . . ,x_k, . . . ,xn)

Example: affine function

y =c0+c1x1+· · ·+ckxk +· · ·+cnxn

Measure of influence ofx_k:

coefficient c_k

Measure of influence of variables

More complicated functions...

y =

n

Y

i=1

x_i^ci

y =

n

X

i=1

c_ix_i² 1/2

y = max

i=1..nmin(ci,xi)

etc.

Measure of influence of variables

A reasonable answer:

−→Approximation of f by an affine function

y =a0+a1x1+· · ·+a_kx_k+· · ·+anxn

Measure of influence ofx_k overf: coefficient a_k

Measure of interaction among variables

What about interactions among variables?

Example: multilinear function

f(x1,x2) =c0+c1x1+c2x2+c12x1x2

Measure ofinteractionbetween x₁ andx₂ withinf: coefficient c₁₂

c₁₂= 0 : zero interaction c12>0 : positive interaction c12<0 : negative interaction

Measure of interaction among variables

LetS ⊆N ={1, . . . ,n}

Measure of interaction among variables{x_k :k ∈S}

−→Approx. of f by a multilinear polynomial of degree6s =|S| fs(x) = X

T⊆N

|T|6s

as(T) Y

i∈T

xi

Measure of interaction among{x_k :k ∈S} inside f: coefficient as(S)

(leading coefficients)

Approximation problem

Multilinear approximation

Denote byMs the set of all multilinear polynomials g: [0,1]ⁿ →R of degree6s

g(x) = X

T⊆N

|T|6s

a_s(T) Y

i∈T

x_i

We define thebest s-th approximationof a function f ∈L²([0,1]ⁿ) as the multilinear polynomialf_s ∈M_s that minimizes

d(f,g)² = Z

[0,1]ⁿ

f(x)−g(x)2

dx among allg ∈Ms

Interaction index

From the bests-th approximation of f, we define the following interaction index(power index ifs = 1)

I(f,S) =as(S) (s =|S|)

= coefficient of Q

i∈S

xi in the best s-th approximation off

In the discrete case, i.e.,f :{0,1}ⁿ→R

Non-weighted distance: Hammer and Holzman (1992) Weighted (multiplicative) distance: M. and Mathonet (2010)

Interaction index

Explicit expression

I(f,S) = 12^s Z

[0,1]ⁿ

f(x) Y

i∈S

x_i −1 2

dx

Properties:

f 7→ I(f,S) is linear and continuous The index I is symmetric

f ∈Mn ⇒ I(f,S) =IB(f|_{0,1}n,S) (Banzhaf index) . . .

Question: How can we see that this index actually measures an interaction?

Interaction index

Interpretation:

Dkf(x) = rate of change w.r.t.xk of f atx

= local contributionof x_k overf atx

D_jD_kf(x) = local interactionbetween x_j andx_k withinf at x

Theorem

Iff is sufficiently smooth, then I(f,S) =

Z

[0,1]ⁿ

q_S(x)D_Sf(x) dx

whereq_S(x) is the p.d.f. of independent beta distributions (2,2)

Interaction index

What iff is not differentiable?

S-derivative→ discrete S -derivative(S-difference quotient) Theorem

We have

I(f,S) = Z

x∈[0,1]ⁿ

Z

hS∈[0,1−x_S]

p_S(h) ∆^S_hf(x) Q

i∈Sh_i dh_S dx wherep_S(h) is a p.d.f. over the domain of integration

Interaction index

Examples:

f(x) = ¹_n

n

P

i=1

xi

I(f,{k}) = ¹_n I(f,S) = 0 s >2

f(x) = n

Q

i=1

xi

1/n

I(f,{k}) = _n+1ⁿ n 6 n+2

I(f,S) = _n+1ⁿ n 6 n+2

s

Interaction index

Further examples:

We have explicit expressions forI(f,S) when f(x) = P

T⊆N

c(T) Q

i∈T

ϕi(xi)

(pseudo-multilinear function) f(x) = P

T⊆N

c(T) min

i∈T x_i

(Choquet integral)

Further properties

A variablex_k is inefficientfor f iff does not depend on x_k f(x_k;x_N\k)−f(0_k;x_N\k) = 0

Property

Ifx_k is inefficient for f then

I(f,S) = 0 ∀S 3k Discrete case: k null player

Further properties

A variablexk is dummy for f if

f(x_k;x_N\k)−f(0_k;x_N\k) =f(x_k;0_N\k)−f(0_k;0_N\k) (the marginal contribution ofx_k is independent of x_N\k)

Discrete case: k dummy player

Further properties

A combination of variables{x_k :k∈S}is dummy forf if f(x_S;x_N\S)−f(0_S;x_N\S) =f(x_S;0_N\S)−f(0_S;0_N\S)

Property

If{x_k :k ∈S}is dummy for f, then

I(f,T) = 0 ∀T such thatT ∩S 6=∅andT \S 6=∅

Discrete case: dummy coalition

Conclusion

Given a functionf ∈L²([0,1]ⁿ), we have defined an index to measure

the influence of variables over f

the interaction among variables within f Interpretations

leading coefficients in multilinear approximation (multilinear regression)

meanS-derivative orS-difference quotient Properties

linearity, symmetry, S-monotonicity...

natural extension of the Banzhaf interaction index (null and dummy players...)

Thank you for your attention!

arXiv : 0912.1547