Robust and accurate inference for generalized linear models

Robust and Accurate Inference for Generalized Linear Models:

Complete Computations

by

Serigne N. Lˆo

The George Institute University of Sydney, Australia

and

Elvezio Ronchetti

Department of Econometrics University of Geneva, Switzerland

February 2008 / Revised: May 2009

APPENDIX A

To determine λ(β), we calculate

−n∂Kψ(λ;β)

∂λ = −

n

X

i=1

∂K_ψⁱ(λ;β)

∂λ

=

n

X

i=1

∂(µiλ^Tx_i+ ^{b(θ0i)−b(θ}_a(φ)^{0i+λTxia(φ))})

∂λ

=

n

X

i=1

[µix_i−b⁰(θ0i+λ^Tx_ia(φ))·x_i]

= 0

Sinceg(·) is the canonical link,θ_i =x^T_i β, and −n^∂2_∂λ∂λ^Kψ(λ;β)T is negative definite, this equation has a unique solution given by λ(β) = ^β−β_a(φ)⁰.

Then, by replacing this expression for λ in Kψ and after simplification we obtain

−K_ψⁱ(λ(β);β) = (θi−θ0i)µi−(b(θi)−b(θ0i))

a(φ) ,

and

h(β) = −K_ψ(λ(β);β)

= 1

n

n

X

i=1

−K_ψⁱ(λ(β);β)

= 1

n

n

X

i=1

(θi−θ_0i)µi−(b(θi)−b(θ0i)) a(φ)

= 1

n

n

X

i=1

b⁰(x^T_i β)x^T_i (β−β₀)−(b(x^T_i β)−b(x^T_i β₀))

a(φ) .

APPENDIX B Calculation of the integrals I_i1, I_i2, I_i3

(i)

I_i1 = Z

ri<−c

e^−λ

Tc ^w(xi)

V1/2(µi)µ⁰_i−λ^T˜a(β)

·e^{yθ0ia(φ)−b(θ0i)} ·e^d(y;φ)·dy

= e^−λ

Tc ^w(xi)

V1/2(µi)µ⁰_i−λ^T˜a(β)

· Z

ri<−c

e^{yθ0ia(φ)−b(θ0i)} ·e^d(y;φ)·dy

= e^−λ

Tc ^w(xi)

V1/2(µi)µ⁰_i−λ^T˜a(β)

· Z

y<−cV^1/2(µi)+µi

e^{yθ0ia(φ)−b(θ0i)} ·e^d(y;φ)·dy

= e^−λ

Tc ^w(xi)

V1/2(µi)µ⁰_i−λ^T˜a(β)

·P(Zⁱ ≤ −cV^1/2(µ_i) +µ_i)

whereZⁱis a random variable distributed according to the exponential family (2) with parameter θ_0i.

(ii)

Ii2 = Z

|ri|<c

e

yλT µ0 i V1/2(µi)

w(xi) V1/2(µi) ·e

−λT µiµ0 i V1/2(µi)

w(xi)

V1/2(µi) ·e^{−λT˜a(β)}·e

yθ0i−b(θ0i)

a(φ) ·e^d(y;φ)·dy

= Z

|ri|<c

e

yλT µ0 iw(xi) V(µi) ·e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

−b(θ0i) a(φ) ·e

yθ0i

a(φ) ·e^d(y;φ)·dy

= Z

|ri|<c

e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e^{−b(θa(φ)0i)} ·e

y(θ0i+λT µ0

iw(xi)a(φ) V(µi) )

a(φ) ·e^d(y;φ)·dy

= Z

|ri|<c

e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

·e

y(θ0i+λT µ0

iw(xi)a(φ)

V(µi) )−b(θ0i+λT µ0 iw(xi)a(φ) V(µi) )

a(φ) ·e^d(y;φ)·dy

= e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

· Z

|ri|<c

e

y(θ0i+λT µ0 iw(xi)a(φ)

V(µi) )−b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )

a(φ) ·e^d(y;φ)·dy

= e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

. P(−cV^1/2(µi) +µ_i < Z_λⁱ < cV^1/2(µi) +µ_i)

whereZ_λⁱ is a random variable distributed according to the exponential family (2) with parameter [θ0i+^λTµ0i_V^w(x_(µiⁱ₎^)a(φ)].

(iii) This result can be easily derived as in (i).

We obtain:

I_i3 =e^λ

Tc ^w(xi)

V1/2(µi)µ⁰_i−λ^T˜a(β)

·P(Zⁱ ≥cV^1/2(µ_i) +µ_i).

APPENDIX C

For i= 1, ..., n, we have from Appendix B:

∂I_i1

∂λ + ∂I_i2

∂λ +∂I_i3

∂λ = −hcw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)i

·Ii1

− hµ_iµ⁰_iw(x_i)

V(µi) + ˜a(β)− µ⁰_iw(x_i)

V(µi) b⁰(θ_0i+λ^Tµ⁰_iw(x_i)a(φ) V(µi) )i

·I_i2 + e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

h ∂

∂λP(|Z_λⁱ |< c)i + hcw(xi)µ⁰_i

V^1/2(µi) −a(β)˜ i

·I_i3

= −hcw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)i

·I_i1

− hµiµ⁰_iw(xi)

V(µi) + ˜a(β)− µ⁰_iw(xi)

V(µi) b⁰(θ0i+λ^Tµ⁰_iw(xi)a(φ) V(µi) )i

·Ii2

+ e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) · µ⁰_iw(xi) V(µi) E^Z_|r^λi

i|<c[Y]

− µ⁰_iw(xi)

V(µi) b⁰(θ0i +λ^Tµ⁰_iw(xi)a(φ) V(µi) )·Ii2

+ hcw(xi)µ⁰_i

V^1/2(µi) −a(β)˜ i

·I_i3

= −hcw(x_i)µ⁰_i

V^1/2(µi) + ˜a(β)i

·I_i1

− hµ_iµ⁰_iw(xi)

V(µi) + ˜a(β)i

·Ii2

+ e

−λT µiµ0 iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0 iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) · µ⁰_iw(xi)

V(µi) E^Z_|r^iλ_i|<c[Y] + hcw(xi)µ⁰_i

V^1/2(µi) −˜a(β)i

·I_i3.

Furthermore,

∂s(λ;β)

∂λ =

n

X

i=1

∂h∂Ii1

∂λ +^∂Ii2_∂λ +^∂Ii3_∂λ Ii1+Ii2+Ii3

i

∂λ

=

n

X

i=1

∂²(Ii1+Ii2+Ii3)

∂λ∂λ^T ·(Ii1+Ii2+Ii3)−[^∂I_∂λⁱ¹ + ^∂I_∂λⁱ² +^∂I_∂λⁱ³]·[^∂I_∂λⁱ¹ + ^∂I_∂λⁱ² +^∂I_∂λⁱ³]^T

(I_i1+I_i2+I_i3)² .

LetS1i and S2i such that:

S1i : = ∂²(Ii1+I_i2+I_i3)

∂λ∂λ^T ·(Ii1+Ii2+Ii3)

= ∂(^∂I_∂λⁱ¹ + ^∂I_∂λⁱ² +^∂I_∂λⁱ³)

∂λ^T ·(Ii1+Ii2+Ii3)

= (Ii1+Ii2+Ii3)·n

£cw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)¤

·£cw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)¤T

·Ii1

+ £µiµ⁰_iw(xi)

V(µi) + ˜a(β)¤

·£µiµ⁰_iw(xi)

V(µi) + ˜a(β)¤T

·Ii2

− £µ_iµ⁰_iw(x_i)

V(µi) + ˜a(β)¤

·hµ_iw(x_i) V(µi)

iT

E_|r^Zλi

i|<c[Y] . e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

− hµ_iµ⁰_iw(xi)

V(µi) + ˜a(β)i

.hµ⁰_iw(xi) V(µi)

iT

.E_|r^Zλi

i|<c[Y] . e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

+ e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) .hµ⁰_iw(xi)i

·hµ⁰_iw(xi)iT

.E_|r^Zλi_|<c[Y²]

+ £cw(x_i)µ⁰_i

V^1/2(µi) −˜a(β)¤

·£cw(x_i)µ⁰_i

V^1/2(µi) −˜a(β)¤T

·I_i3o

and

S2i : = £∂I_i1

∂λ +∂I_i2

∂λ + ∂I_i3

∂λ

¤·£∂I_i1

∂λ +∂I_i2

∂λ + ∂I_i3

∂λ

¤T

= £cw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)¤

·£cw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)¤T

·I_i1² + £µ_iµ⁰_iw(xi)

V(µi) + ˜a(β)¤

·£µ_iµ⁰_iw(xi)

V(µi) + ˜a(β)¤T

·I_i2² + e^{−2λT µiµ}

0iw(xi)

V(µi) ·e^{−2λT˜a(β)}·e

2b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−2b(θ0i)

a(φ)

·£µ⁰_iw(xi) V(µ_i)

¤·£µ⁰_iw(xi) V(µ_i)

¤T

.£

E_|r^Z_i^λi_|<c[Y]¤2

+ £cw(xi)µ⁰_i

V^1/2(µi) −a(β)˜ ¤

·£cw(xi)µ⁰_i

V^1/2(µi) −a(β)˜ ¤T

·I_i3² + 2·£cw(xi)µ⁰_i

V^1/2(µ_i) + ˜a(β)¤£µiµ⁰_iw(xi)

V(µ_i) + ˜a(β)¤T

Ii1Ii2

− 2·£cw(xi)µ⁰_i

V^1/2(µi) + ˜a(β)¤

·hµ⁰_iw(xi) V(µi)

iT

.E_|r^Zλi

i|<c[Y]·I_i1 . e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

− 2£cw(xi)µ⁰_i

V^1/2 −˜a(β)¤

·£cw(xi)µ⁰_i

V^1/2 + ˜a(β)¤T

·I_i1·I_i3

− 2·£µ_iµ⁰_iw(x_i)

V(µi) + ˜a(β)¤

·hµ⁰_iw(x_i) V(µi)

iT

.E_|r^Zλi

i|<c[Y]·I_i2

·e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ)

− 2·£µ_iµ⁰_iw(xi)

V(µi) + ˜a(β)¤£cw(xi)µ⁰_i

V^1/2(µi) −˜a(β)¤T

I_i2·I_i3 + 2·£cw(xi)µ⁰_i

V^1/2(µ_i) −˜a(β)¤

·hµ⁰_iw(xi) V(µ_i)

iT

.E_|r^Z_i^iλ_|<c[Y]·Ii3

·e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) .

Then,

∂s(λ;β)

∂λ =

n

X

i=1

[S1_i −S2_i] (Ii1+I_i2 +I_i3)²

=

n

X

i=1

[µ⁰_i·µ^0T_i ]

(Ii1+I_i2 +I_i3)²w²(x_i)n

£ c

V^1/2(µi)− µi

V(µi)

¤2

·I_i1I_i2+£

2 c

V^1/2(µi)

¤2

I_i1I_i3

+ £ c

V^1/2(µi)+ µ_i V(µi)

¤2

·I_i2I_i3 + 2·e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λTa(β)˜} ·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) · 1

V(µi) . £

( c

V^1/2(µ_i)− µi

V(µ_i))·Ii1−( c

V^1/2(µ_i) + µi

V(µ_i))·Ii3

¤·E_|r^Zλ_iⁱ_|<c[Y]

+ e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0 iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) · 1

V²(µi) . £

I_i1+I_i2+I_i3¤

·E_|r^Zλi

i|<c[Y²]

− £

e^{−λT µiµ}

0iw(xi)

V(µi) ·e^{−λT˜a(β)}·e

b(θ0i+λT µ0

iw(xi)a(φ) V(µi) )−b(θ0i)

a(φ) ¤2

· 1

V²(µi)·£ E_|r^Zλi

i|<c[Y]¤2o .

APPENDIX D

Special cases

(i) Y_i ∼N(µ_i, σ²)

b(θi) = ^θ₂²ⁱ a(φ) =σ²

and in this case ˜a(β) = 0. Then, we have :

∂s(λ;β)

∂λ =

n

X

i=1

x_ix^T_i · w²(xi) (Ii1+I_i2+I_i3)²

n¡

c−x^T_i β¢2

·I_i1I_i2 + ¡

2.c¢2

·Ii1Ii3+¡

c+x^T_i β¢2

·Ii2Ii3

+ 2·e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2} . £

(c−x^T_i β)I_i1−(c+x^T_i β)·I_i3¤

·E_|r^Zλi

i|<c[Y]

+ e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2} ·[Ii1+I_i2+I_i3]·E_|r^Ziλ

i|<c[Y²]

− £

e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2}¤2

·£ E_|r^Ziλ

i|<c[Y]¤2o

=

n

X

i=1

x_ix^T_i ·A_i(λ),

where A_i(λ) is scalar function defined by

A_i(λ) = w(xi)

(Ii1+I_i2+I_i3)² · n ¡

c−x^T_i β¢2

·I_i1I_i2+¡ 2.c¢2

·I_i1I_i3+¡

c+x^T_i β¢2

·I_i2I_i3 + 2·e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2}

. £

(c−x^T_i β)Ii1−(c+x^T_i β)·Ii3

¤·E_|r^Z_i^λi_|<c[Y]

+ e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2} ·[Ii1+Ii2+Ii3]·E_|r^Zλ_iⁱ_|<c[Y²]

− £

e^{xTiλw(xi)xTi(2β0−β)+(xTiλw(xi)σ)2}¤2

·£ E_|r^Zλi

i|<c[Y]¤2o .

(ii) Y_i ∼P(µi)

b(θ) = e^θ, a(φ) = 1 Then, we have :

∂s(λ;β)

∂λ =

n

X

i=1

xix^T_i · w²(xi)·e^2xTiβ (Ii1+Ii2+Ii3)²

n¡

ce^−12xTiβ−1¢2

·Ii1Ii2

+ ¡

2.ce^−12xTiβ¢2

·I_i1I_i3+¡

ce^−12xTiβ+ 1¢2

·I_i2I_i3

+ 2·e^{−xTiλwexTi β−λT˜a(β)}.e^{[exTi(β0+w(xi)λ)−exTi β0]}·e^−xTiβ ·E_|r^Zλi

i|<c[Y]

·£¡

ce^−12xTiβ −1¢

·Ii1−¡

ce^−12xTiβ + 1¢

·Ii3

¤

+ e^{−xTiλwexTi β−λT˜a(β)}.e^{[exTi(β0+w(xi)λ)−exTi β0]}.e^−2xTiβ[Ii1+Ii2+Ii3]·E_|r^Z_i^iλ_|<c[Y²]

− £

e^{−xTiλwexTi β−λT˜a(β)}.e^{[exTi(β0+w(xi)λ)−exTi β0]}¤2

.e^−2xTiβ·³

E_|r^Z_i^λi_|<c[Y]´2o

=

n

X

i=1

xix^T_i ·Ai(λ),

where Ai(λ) is scalar function defined by

Ai(λ) = w²(xi)·e^2xTiβ (I_i1+I_i2+I_i3)² n ¡

ce^−12xTiβ −1¢2

·I_i1I_i2+¡

2.ce^−12xTiβ¢2

·I_i1I_i3+¡

ce^−12xTiβ + 1¢2

·I_i2I_i3 + 2·e^{−xTiλwexTi β−λT˜a(β)}.e^{[exTi(β0+w(xi)λ)−exTi β0]}·e^−xTiβ·E_|r^Zλi

i|<c[Y]

·£¡

ce^−12xTiβ−1¢

·Ii1−¡

ce^−12xTiβ+ 1¢

·Ii3

¤

+ e^{−xTiλwexTi β−λTa(β)˜} .e^{[exTi(β0+w(xi)λ)−exTi β0]}.e^−2xTiβ[Ii1+Ii2 +Ii3]·E_|r^Z_i^λi_|<c[Y²]

− £

e^{−xTiλwexTi β−λT˜a(β)}.e^{[exTi(β0+w(xi)λ)−exTi β0]}¤2

.e^−2xTiβ ·³

E_|r^Zλ_iⁱ_|<c[Y]´2 o .

(iii) Yi ∼Bin(m, πi)

b(θ) = m·log(1 +e^θ), a(φ) = 1 Then, we have :

∂s(λ;β)

∂λ =

n

X

i=1

x_ix^T_i w²(x_i)e^2xTiβ (Ii1 +I_i2+I_i3)²

n¡ c−√

m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i1I_i2

+ ¡ 2c

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i1I_i3+¡ c+√

m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i2I_i3

+ 2.¡1 +x^T_i β0+x^T_iλw(xi) 1 +β₀^Tx_i

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 me^xTiβ . £¡ c−√m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢I_i1−¡ c+√m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢I_i3¤

·E_|r^Zλi

i|<c[Y] + ¡1 +x^T_i β₀+x^T_i λw(xi)

1 +x^T_i β0

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 m²e^2xTiβ . £

Ii1 +Ii2+Ii3

¤·E_|r^Zλ_iⁱ_|<c[Y²]

− ¡1 +x^T_i β₀+x^T_i λw(xi) 1 +x^T_i β₀

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 m²e^2xTiβ

£E_|r^Zλ_iⁱ_|<c[Y]¤2o

=

n

Xxix^T_i ·Ai(λ),

where Ai(λ) is scalar function defined by

A_i(λ) = w²(xi)e^2xTiβ (Ii1+Ii2+Ii3)²

n¡ c−√m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i1I_i2

+ ¡ 2c

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i1I_i3+¡ c+√m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢2

I_i2I_i3

+ 2.¡1 +x^T_i β₀+x^T_i λw(xi) 1 +x^T_i β0

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 me^xTiβ . £¡ c−√

m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢Ii1−¡ c+√

m·e^12xTiβ

√m·e^12xTiβ(1 +e^xTiβ)

¢Ii3

¤·E_|r^Zλ_iⁱ_|<c[Y]

+ ¡1 +x^T_i β0+x^T_i λw(xi) 1 +x^T_i β₀

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 m²e^2xTiβ . £

I_i1+I_i2+I_i3¤

·E_|r^Ziλ

i|<c[Y²]

− ¡1 +x^T_i β₀+x^T_i λw(x_i) 1 +x^T_i β0

¢m

.e

−mxTi λw(xi)exTi β 1+exTiβ

.e^{−λT˜a(β)}· 1 m²e^2xTiβ

£E_|r^Zλi

i|<c[Y]¤2o .