Generalized linear latent variables models: Estimation, inference and empirical analysis of financial data

Thesis

Reference

Generalized linear latent variables models: Estimation, inference and empirical analysis of financial data

HUBER, Philippe

Abstract

Generalized Linear Latent Variable Models (GLLVM), as defined in Bartholomew and Knott (1999), enable the modelling of relationships between manifest and latent variables. They extend the structural equation modelling techniques, which are powerful tools in the social sciences. However, because of the complexity of the log-likelihood function of a GLLVM, an approximation such as numerical integration must be used for estimation and inference.

Depending on the choice of the approximation, the estimators can be biased, and/or their computation can be extremely time consuming. In this work, we propose a new estimator for the parameters of a GLLVM, the LAMLE, based on a Laplace approximation to the likelihood function and which can be computed even for models with a large number of variables. The LAMLE can be viewed as an M-estimator, leading to readily available asymptotic properties, correct inference and an Akaike Information Criterion as a model selection criterion. We introduce e new software called L-Cube that computes the LAMLE and propose new algorithms to increase the computation speed. In particular, we propose a [...]

HUBER, Philippe. Generalized linear latent variables models: Estimation, inference and empirical analysis of financial data . Thèse de doctorat : Univ. Genève, 2004, no. SES 564

URN : urn:nbn:ch:unige-83861

DOI : 10.13097/archive-ouverte/unige:8386

Available at:

http://archive-ouverte.unige.ch/unige:8386

Disclaimer: layout of this document may differ from the published version.

1 / 1

!

"

#! $ ! !

% & '

( ) *+,

& -..,

0 4 50

1 3 00 1 3 1 0

4 6 2-..,

/

7//7)

/ / $ # 8//$#9 : ;

<8=6669

( > 1 1

> !'

//$# >

>

? > @ '

//$#/7#// >

'

( /7#/

M

^!

7'' @ A

B /7#/

@ 1

: C A

7

QR

LLD

^B

:

p

^!:

7 > : /7#/

/' 7

//$# ' 1

' ' '

1

/45 00 08//$#90: ;

< 8=6669 0 !

@00 0 01 1

0 A ! 3

//$#0 4> >0 10 3

30 3 > D 0 ?

00 D 4

//$# /7#/ 0 3 > /

! D 0 4

/ /7#/ D 0 0

M

^{00 1 30 4 0}

4 37'' 02500 00

) 0 1 /7#/ !

> 0

01 1

0 3C

0

QR

¹

LLD

⁾

0 0:1

p

^{! 3}

: 4 4

0 > 00 /7#/

0 13 >

A 3 3 //$#

)

13 1 1 0

00

3 0 > 2> 0

0E 0 F0 30

0 3 3 >4

3 300 5

@ ' ' # ! $ !

' !

@

: !

@ ' %

# ' 3 ' !

' 2

/7

#' # :

: # @ 3

# < B

'

@ A

>

@ '

7 @ ' ) @

' @ '

-= / # *

-== ) +

-=- ; +

-=G " H

-- / I

--= I

--- / / $ # 6

--G ! J 8J9 > ==

--, #/ J > =-

G= B / >K =,

G- # =*

G-= / > =*

G-- /7 > #> /' =+

GG # =I

GG= 7 > =I

GG- /7 > #> /' =6

G, 7 =6

G,= ) -.

G,- ; -=

G,G " -G

G* -G

!

,= @ -H

,== )1 /7#/ -H

,=- -I

,- A -6

,-= "1 -6

,-- " 1 G=

,-G G=

"

*= 7 /7#/ GG

*- # GG

*G ( G,

*, ; G*

** G+

**= A: @ G+

**- # 7@A GH

# $

+= #> ,.

+- ,=

+-= " ,-

+-- "1 ,G

+-G 7 ,,

+G 7 ,*

+G= A > /7#/ ,*

+G- ( > ,+

+GG ( > ,+

+, ,H

! % &

H= / *-

H- > *G

HG # *,

HG= *,

HG- *,

HGG # **

HG, *+

HG* *I

H, A ++

' #(

I= +6

I- H.

) **+ !

7= 1 H-

7== ! 1 HG

7- ( / HG

, %-* " . !

;= /7#/ H*

;== /!' H*

;=- 1

ˆ z i(2)

^H+

;=G HH

;=, F

ˆ z i(2)

^HH

;- /7#/ HI

;-= /!' HI

;-- 1

ˆ z i(2)

^H6

;-G H6

;-, F

ˆ z i(2)

^H6

;G /7#/ I.

;G= /!' I.

;G- 1

ˆ z i(2)

^I=

;GG F

ˆ z i(2)

^I-

;, > !' I-

% " " " . + '

% / 0) . '

1 "222 '!

= (

M

^{! IH}

-

M

^{! II}

-= A II

-- 7 II

G # I6

G= ( 7''@ A 87@A9 I6

G- ( 7''@ A 87@A 9 6.

, ( 6.

,= ( 7@A 6.

,- ( 7@A 6=

3 4. 0 * 5

BC _a

^{6 (}

=

z ₀

a

^6G

@ : : : '

(

@ >

8 7 <=6IH9

F @

(

'

@ > K K

'

( C

8 9

18 9

( 1 C '

1 8 9 K

( (

!' 1 B

1 8 9

(

7 (

' 8 9

8 9 #> ' 8

,=-9 ' &L ' 8=6+69

>

1 #

'# 8#

#-..=97#"8 9 J8; =6I+9

8&L ' =66.9

7

F ' : @

B8 >9

' 8

--=9 @

>

> B A -

'

M

8

>9 > 1

>

(

> >

@

(

'

( : ;

8=6I,9 # #' 8=66+ -...9

#' < 8-...9 >

8//$#9 '

8 ---9

//$# M @

'

1

8 --G9 #' 8=66+ -...9 #' < 8-...9

! J 8J9 J >

' //$# 8 G*9

7 1 8 !'

' ' -..-9 ( 1

: > :

8 +,9

B / > ' (

@ >

//$#: / >

; A 8=66G9 / ; 8=66+9

( / > '

1 >

# 8

H=9

>#A#A #A#E N # 8=66+9 B

>

'

( ' @ A - >

C 8

9 #08=6HI9&L ' L 8=66G9

7 <O 8=6II9//$#B

F //$#> ;

#' @ ' P

/ # : B

! J >

@A G //$#B

>! J

/ > > !'

'

#'8=66+ -...9 #' <8-...9 '

B

/ >!':

1 ( 8 / > >

' /7#/9 > '

>

M

^!

8=6+,9 :

: J

/ >

@ '

1 ( //$# !

> !'

A ,=- '!

1

( /7#/

M

^{! !}

@A * /7#/

B 7''

M

^{! 8 =66H9}

' ! : 8=66,9

B 1

'

> /7#/

A +

( //$# 1

M B

J)! >

)

> !'

@ :

1 ' >

> !' F

: >

(

/7#/

@ A H ( : :

' '

F ' B //$#

1 ' 3 B

(

1

7 > ( :

@ //$# B

>

//$#

: 1 F

-- @ > #' 8=66+-...9

#'<8-...9&L '8=66.9 > "

//$#

! J 8J9 > #'

/ #8/#9//$#

8 '#A) =6I69 @

'/#

' ' //$# (

x

q

^>

z ^(k)

:> 7 '

x

z ^(k)

^:

>

x

z = [1 z ⁽¹⁾ . . . z ^(q) ] ^T

^>

8 9

7 /# C

= (

x

z

^{> C}

g(x |z ) =

^>

xu( α ^T z ) − b(u( α ^T z ))

φ + c(x, φ)

8-=9

u( α ^T z )

b(u( α ^T z ))

c(x, φ)

^:

g(x |z )

φ

8 -== -=G > 9

- ( >

u

^C

u( α ^T z ) = u

α ₀ +

q j=1

α _j z ^(j)

α = [α ₀ , . . . , α _q ] ^T

G ( ' >

x |z

^'

ν

^C

ν(E(X |z )) = ν(µ) = α ^T z .

^8--9

)

u

^'

7 > !

@

> B P

>

B :

b

c

φ

^!

18-=9

/

X |Z = z

µ

σ ²

⁽

'

ν(µ) = µ = α ^T z .

(

φ = σ ²

^B

b( α ^T z ) = [ α ^T z ] ² /2

^8-G9

c(x, φ) = − 1 2

x ²

φ +

(2πφ)

.

^8-G9

g(x |z ) = 1

√ 2πφ

>

− 1

2φ (x − α ^T z ) ²

.

^8-,9

/

X |Z = z

0, 1, . . . , m

^>

m · π

^{( '}

ν(π) =

mπ m − mπ

=

π 1 − π

=

(π) = α ^T z .

π := π( z ) =

^>

( α ^T z ) 1 +

^>

( α ^T z ) .

( 8

φ = 1

⁹

b

c

b( α ^T z ) = m

(1 +

^>

( α ^T z ))

^8-*9

c(x, φ) = c(x) =

m x

.

^8-*9

B

g(x |z ) = m

x

π( z ) ^x (1 − π( z )) ^(m−x) .

^8-+9

/

X |Z = z

^{8 9}

=

m

^{( '}

ν(γ _s ) = log

γ _s 1 − γ _s

= α ^T _s z ,

γ _s

^:

X |z

s

s = 1 . . . m

⁽

s

^>

α s

^:

α _0s

α

" C

−∞ < α ₀₁ < α ₀₂ < . . . < α _0m = ∞ .

(

γ _s := γ _s ( α ^T _s z ) = exp( α ^T _s z ) 1 + exp( α ^T _s z ) .

B 1 =!

'

C

u _s ( α ^T _s z ) = log

γ _s γ _s+1 − γ _s

.

^8-H9

b(u _s ( α ^T _s z ))) = log(1 + exp(u _s ( α ^T _s z ))) = log

γ _s+1 γ _s+1 − γ _s

8-I9

c(x, φ) = 0.

^8-I9

B

g(x |z ) = m s=1

(γ _s − γ _s−1 ) ^{x s} ,

^8-69

x _s = 1

s

x _s = 0

^{( 1}

g(x |z ) =

m−1

s=1

γ _s γ _s+1

_x ^∗ _s

γ _s+1 − γ _s γ _s+1

_x ^∗ _{s+1 −x} ^∗ _s

= exp _m−1

s=1

x ^∗ _s u _s ( α ^T _s z ) − x ^∗ _s+1 b(u _s ( α ^T _s z ))

,

^8-=.9

x ^∗ _s = 1

x ≤ s

x ^∗ _s = 0

^{( '}

X

^{* = *}

x

^{' G (}

x _s = 0

s = 1, 2, 4

^*

x ₃ = 1

x ^∗ _s = 0

s ≤ 3

x ^∗ _s = 1

7 F

#0 8=6HI9&L '8=66.9 7 <O 8=6II9 @

(

#'8=66+9 #' <8-...9 > ; 3

' 8; =6I,9 @

(

@

( / >

X

^;

z

α

^>

/

Y

z

α ^T z

z

^'

X

Y

X

^{' =}

Y

^{. (}

E(X |z ) = P (Y > 0 |z ) = Φ( α ^T z ),

Φ( · )

^{B 1}

(E(X |z )) = α ^T z .

^8-==9

@ ' 8

>!' 8-==99

8 &L ' =6+6 =66.9 !

1 !

@ >

•

^{8 =6.=9}

•

^C

/ 8=6IH9E # 8=6I,9E/ ; 8=66-9

•

^{C /}

8=6IH9E# 8=6I,9

( (

>

1

( !

//$# 8 9 > '

( ' 8

| Φ(x) − Ψ(1.7x) | < 0.01 ∀ x

Ψ

^{/ )' 8=6+I99}

8 +,9

# 8

9 ( F

' 1 8 9

(

'

@ //$# ' /# 8#A

) =6I69 ( //$#

p

x ^(j)

j = 1, . . . , p

q < p

z ^(k)

k = 1, . . . , q

^@

>

g _j (x ^(j) |z )

j = 1, . . . , p

>

g _j (x ^(j) |z ) =

^>

(x ^(j) u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z )))/φ _j + c _j (x ^(j) , φ _j )

8-=-9

z = [1, z ⁽¹⁾ , . . . , z ^(q) ] ^T = [1, z ^T ₍₂₎ ] ^T

g _j

x ^(j)

α j = [α _j0 , . . . , α _jq ] ^T

⁸

9

φ _j

( //$#

@ >

2

_p

j=1 g _j (x ^(j) |z )

⁾

@

h( z (2) )

> 8 GG9 ( 2

p j=1

g _j (x ^(j) |z )h( z (2) ).

^8-=G9

z (2)

8-=G9 "

f _α,φ ( x ) =

p j=1

g _k (x ^(j) |z )

h( z (2) ) dz (2) .

^8-=,9

)

g _j (x ^(j) |z )

j

> "

α _j

φ _j

j = 1 . . . p

⁽

x

z

) 8-=,9 8!

/ =6HH9 > > 8-=,9

3 > > ( #

: > > '

α

φ

> / 8=66H9 )

> #

/

n

x 1 , . . . , x n

x i = [x ⁽¹⁾ _i , . . . , x ^(p) _i ]

^/

α

(q + 1) × p

^>

α = [ α ₁ , . . . , α _p ]

φ = [φ ₁ , . . . , φ _p ]

!'

l( α , φ|x )

^C

l( α , φ|x ) =

n i=1

f _α,φ ( x i )

= n

i=1

_∞

−∞

p j=1

>

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

h( z (2) ) dz (2) .

^8-=*9

(>' 8#/9 '

F 8-=*9

α ₁ , . . . , α p

φ

^C

∂l( α , φ|x )

∂α _kl = n

i=1

1 f _α,φ ( x i )

∂f _α,φ ( x i )

∂α _kl

= n

i=1

1 f _α,φ ( x i )

_∞

−∞

p j=1

g _j (x ^(j) _i |z )

h( z (2) ) ∂

∂α _kl

x ^(j) _i u _j ( α ^T _j z )

φ _j − b _j (u _j ( α ^T _j z )) φ _j

dz (2) .

^8-=+9

(

k

l

k ^th

l ^th

k = 1, . . . , p

l = 1, . . . , q

∂l( α , φ|x )

∂φ _k =

= n

i=1

1 f _α,φ ( x i )

_∞

−∞

p j=1

g _j (x ^(j) _i |z ) h( z (2) )

− x ^(k) _i u _k ( α ^T _k z ) − b _k (u _k ( α ^T _k z ))

φ ² _k + ∂c _j (x ^(j) _i , φ _j )

∂φ _k

dz (2)

8-=H9

18-=*98-=+98-=H9 >

>

g _j (x ^(j) |z )

^A1

> ( > F

' #' A G

! " #!"$ %

#' 8=66+9 ! J >

8-=+9 8-=H9 (

#

J > 7> 7= ( >

s

¹

1 8-=+9 8-=H9

s

z _t

⁸

t = 1, . . . , s

^{9 > >}

h(z _t )

' /

z t = [1 z _t ]

h(z _t |x i )

z _t

x i

8-=+9 C

∂l( α , φ|x )

∂α _kl

s t=1

h(z _t ) n

i=1

x ^(k) _{i p}

j=1 g _j (x ^(j) _i | z _t ) f _α,φ ( x i )φ _k

∂u _j ( α ^T _j z t )

∂α _kl

− n

i=1

_p

j=1 g _j (x ^(j) _i | z _t ) f _α,φ ( x i )φ _k

∂b _j (u _j ( α ^T _j z t ))

∂α _kl

= s

t=1

[r _1kt ∂u _j ( α ^T _j z t )

∂α _kl − N _t b _k (u _k ( α ^T _k z t )) ]/φ _k ,

^8-=I9

b _j (u _j ( α ^T _j z t )) = ∂b _j (u _j ( α ^T _j z t ))

∂α _kl ,

r _1kt = h(z _t ) n

i=1

x ^(k) _i p j=1

g _j (x ^(j) _i | z _t )

/f _α,φ ( x i )

= n

i=1

x ^(k) _i h(z _t |x i ).

^8-=69

N _t = h(z _t ) n

i=1

p j=1

g _j (x ^(j) _i | z _t )

/f _α,φ ( x i )

= n

i=1

h(z _t |x i ).

^8--.9

( > 8-=H9

∂l( α , φ|x )

∂φ _k

s t=1

− r _1kt u _k ( α ^T _k z t )/φ ² _k + N _t b _k (u _k ( α ^T _k z t ))/φ ² _k + N _t ∂c _k (x ^(k) _i , φ _k )

∂φ _k

.

^8--=9

( > > #/

8-=I9 8--=9

& ' !" %

@

α

^{8-=I9 1}

s t=1

[r _1kt ∂u _k ( α ^T _k z t )

∂α _kl − N _t b _k (u _k ( α ^T _k z t )) ] = 0.

^8---9

(

b _k (u _k ( α ^T _k z t ))

^{: -== -=- -=G (}

x i

^C

)7

x ^(j) ∼ N ( α ^T _k z t , φ _k )

∂b _k (u _k ( α ^T _k z t ))

∂α _kl = z _t ^(l) u _k ( α ^T _k z t )

,7

x ^(j) ∼ B(m, π _k ( z t ))

∂b _k (u _k ( α ^T _k z t ))

∂α _kl = z _t ^(l) π _i (z _t ).

z _t ⁽⁰⁾ = 1

z _t ⁽¹⁾ = z _t

l

^{' . =}

@

∂u _k ( α ^T _k z t )

∂α _kl = z ^(l) _t .

φ _k

^{1 8--=9}

c _k (x ^(k) _i , φ _k )

^-==C

∂l( α , φ|x )

∂φ _k

n i=1

1 f _α,φ ( x i )

s t=1

h( z t ) p j=1

g _j (x ^(j) _i | z _t )

− x ^(j) _i α ^T _k z t − b _k ( α ^T _k z t )

φ ² _k + c _k (x ^(k) _i , φ _k )

8--G9

c _k (x ^(k) _i , φ _k ) = ∂c _k (x ^(k) _i , φ _k )

∂φ _k = 1 2

x ^(k) _i

₂ φ ² _k − 1

φ _k

.

; 1 8--G9 1

φ ˆ _k

φ ˆ _k = 1 _s

t=1 N _t s

t=1

[r _2kt − 2 ˆ α _k0 r _1kt − 2 ˆ α _k1 z _t r _1kt

+( ˆ α _k0 + ˆ α _k1 z _t ) ² N _t ]

^8--,9

r _2kt = n

i=1

x ^(k) _i

₂

h( z t |x _i )

^8--*9

r _1kt

N _t

^{: 8-=69 8--.9}

1

>' !'' '

8-..-9 J1

1 1 ' >

' ( 1

+,

@ A - //$#

>' J > @ :

> 7 / >

J B / >!

> ' 8/7#/9 //$#

B : !

7 --G J >

7 J

F 'C : 1 >

1

s

^>

q

⁷¹

J >

1

' M

(

G*

B / > !

>'

p ⁻¹

p

⁽

1 >

@ / >

!'

/7#/

/ > //$# !

/7#/ #/J

8 G*9

z = [1, z ⁽¹⁾ , . . . , z ^(q) ] ^T = [1, z ^T ₍₂₎ ] ^T

^{; !'}

8' 9' >

> 1 8-=+9

8-=H9 (

x i

f _α,φ ( x i ) = _∞

−∞

p j=1

g _j (x ^(j) _i |z )

h( z (2) ) dz (2)

= _∞

−∞

p j=1

>

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

h( z (2) ) dz (2)

^8G=9

h( z (2) )

µ = 0

Σ = I q

^/

/ >

% (

; / >87-9 >

8G=9 C

f _α,φ ( x i ) = _∞

−∞

p j=1

>

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

h( z ₍₂₎ ) dz ₍₂₎

= (2π) ^−q/2 _∞

−∞

>

p 1

p _p

j=1

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

− z ^T ₍₂₎ z (2)

2

dz (2)

= (2π) ^−q/2 _∞

−∞

e ^{pQ(α,z,x i )} dz (2) ,

^8G-9

Q( α , z , x i ) = 1 p

_p

j=1

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

− z ^T ₍₂₎ z (2)

2

.

^8GG9

7

Q( α , z , x i )

^{: 1 8 7> 7-9 !}

>87-9 8G-9C

f _α,φ ( x i ) = p ^−q/2

( −V ( ˆ z i(2) )) ^−1/2 e ^{pQ(α,ˆz i(2) ,x i )} (1 + O(p ⁻¹ )).

ˆ z i(2)

^>

Q( α , z , x i )

Q ( α , z , x i )

^:

1

ˆ

z i(2) := ˆ z i(2) ( α , φ , x i ) = p

j=1

1 φ _j

x ^(j) _i α _j(2) − ∂b _j ( α ^T _j z )

∂ z i(2)

z=ˆz i

,

^8G,9

α _j(2)

^{: '}

z (2)

α _j = [α _j0 , α ^T _j(2) ] ^T

⁽

n ˆ z i(2)

8 9

( >

Q( α , z , x i )

V ( z ) = ∂ ² Q( α , z , x i )

∂ z ^T ₍₂₎ ∂ z (2)

= − 1 p

p j=1

1 φ _j

∂ ² b _j (u _j ( α ^T _j z ))

∂ z ^T ₍₂₎ ∂ z (2) − 1 p I q

= − 1

p Γ ( α , φ , z ),

Γ ( α , φ , z ) = p

j=1

1 φ _j

∂ ² b _j (u _j ( α ^T _j z ))

∂ z ^T ₍₂₎ ∂ z (2) + I q ,

^8G*9

I q

^>

q × q

⁽

f _α,φ ( x i )

^>

f ˜ _α,φ ( x i ) =

Γ ( α , φ , ˆ z i ) _−1/2

>

p j=1

x ^(j) _i u _j ( α ^T _j ˆ z i ) − b _j (u _j ( α ^T _j ˆ z i ))

φ _j + c _j (x ^(j) _i , φ _j )

− ˆ z ^T _i(2) ˆ z i(2)

2

,

^8G+9

ˆ z i = [1 ˆ z _i(2) ]

q × q

^>

Γ ( α , φ , ˆ z i )

)% % * '

>

f ˜ _α,φ ( x i )

^{: >>!'}

˜ l( α , φ|x , ˆ z ) = n

i=1

f ˜ _α,φ ( x i ).

B

f ˜ _α,φ ( x i )

^{> 1 8G+9}

˜ l( α , φ|x , ˆ z ) = n

i=1

f ˜ _α,φ ( x i )

= n

i=1

− 1

2

Γ ( α , φ , ˆ z i )

+ p

j=1

x ^(j) _i u _j ( α ^T _j ˆ z i ) − b _j (u _j ( α ^T _j ˆ z i ))

φ _j + c _j (x ^(j) _i , φ _j )

− ˆ z ^T _i(2) ˆ z i(2)

2

8GH9

(

α

φ

¹

> F

∂

∂ξ

B =

B ^{−1 ∂B} _∂ξ

∂B

∂ξ

B

F

ξ

α

∂ ˜ l( α , φ|x , ˆ z )

∂α _kl =

n i=1

− 1 2

Γ ( α , φ , ˆ z i ) ⁻¹ ∂ Γ ( α , φ , ˆ z i )

∂α _kl

+ 1 φ _k

x ^(k) _i − ∂b _k ( α ^T _k z )

∂ α ^T _k z

z=ˆz i

ˆ z _il

= 0, k = 1, . . . , p, l = 1, . . . , q.

8GI9

( /7#/

φ

∂ ˜ l( α , φ|x , ˆ z )

∂φ _k =

n i=1

− 1 2

Γ ( α , φ , ˆ z i ) ⁻¹ ∂ Γ ( α , φ , ˆ z i )

∂φ _k

− 1

φ ² _k (x ^(k) _i α ^T _k ˆ z i + b _i ( α ^T _k ˆ z i )) + ∂c _k (x ^(k) _i , φ _k )

∂φ _k

= 0, k = 1, . . . , p. .

^8G69

8GI9 8G69

p(q + 2)

¹

@ 1 8GI9 8G69

n i=1

ψ( x i , θ ˆ ) = 0

θ ˆ = [(

α ˆ ) ^T , φ ˆ ^T ] ^T

θ ˆ

M

^{! :}

8=6+, =6I=9 ( /7#/

8 7 -= --9 @

8 *-9 8 *G9

M

^!

) ' #!A 1 8 N # =66+9

/ > 2

˜ l( α , φ|x , ˆ z )

¹

@

( 2:

8 ; < =6669

( ' 1!

'

)% (

/

z (2)

^>

R

⁽

x i

^8G-9

f _α,φ ( x i ) = _∞

−∞

p j=1

>

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

h( z (2) ) dz (2)

= (2π) ^−q/2 det( R ) ^−1/2 _∞

−∞

>

p 1

p _p

j=1

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

− z ^T ₍₂₎ R ⁻¹ z ₍₂₎ 2

dz (2)

= (2π) ^−q/2 det( R ) ^−1/2 _∞

−∞

e ^{p·Q(α,z,x k ,R)} dz (2) ,

^8G=.9

Q( α , z , x i , R ) = 1 p

_p

j=1

x ^(j) _i u _j ( α ^T _j z ) − b _j (u _j ( α ^T _j z ))

φ _j + c _j (x ^(j) _i , φ _j )

− z ^T ₍₂₎ R ⁻¹ z (2)

2

.

7 G- / > 87-9 8G=.9

>

f ˜ _α,φ ( x i ) =

Γ ( α , φ , R , ˆ z i ) _−1/2

det( R ) ^−1/2

>

p j=1

x ^(j) _i u _j ( α ^T _j ˆ z i ) − b _j (u _j ( α ^T _j ˆ z i ))

φ _j + c _j (x ^(j) _i , φ _j )

− ˆ z ^T _i(2) R ⁻¹ ˆ z i(2)

2

,

^8G==9

Γ ( α , φ , R , z ) = p

j=1

1 φ _j

∂ ² b _j (u _j ( α ^T _j z ))

∂ z ^T ₍₂₎ ∂ z (2)

+ R ⁻¹ ,

^8G=-9

ˆ z i = [1 ˆ z i(2) ]

ˆ

z i(2) := ˆ z i(2) ( α , φ , R , x i ) = p

j=1

1 φ _j

x ^(j) _i − ∂b _j ( α ^T _j z )

∂ α ^T _j z z=ˆz i

Rα j(2) .

^8G=G9

)% % * '

" G- !

'

˜ l( α , φ , R|x , ˆ z )

^{8G==98G=-9 8G=G9C}

˜ l( α , φ , R|x , ˆ z ) = n

i=1

− 1

2

Γ ( α , φ , R , ˆ z i )

− 1

2 log det( R ) +

p j=1

x ^(j) _i u _j ( α ^T _j ˆ z i ) − b _j (u _j ( α ^T _j ˆ z i ))

φ _j + c _j (x ^(j) _i , φ _j )

− ˆ z ^T _i(2) R ⁻¹ ˆ z i(2)

2

.

^8G=,9

(

ˆ σ _kl

σ _kl

^>

R

k, l = 1 . . . q

∂ ˜ l( α , φ , R|x , ˆ z )

∂σ _kl =

n i=1

− 1 2

Γ ( α , φ , R , ˆ z i ) ⁻¹ ∂ Γ ( α , φ , R , ˆ z i )

∂σ _kl

− 1 2

R ⁻¹ ∂ R

∂σ _kl

+ 1

2 ˆ z ^T _i(2) R ⁻¹ ∂ R

∂σ _kl R ⁻¹ ˆ z i(2)

,

^8G=*9

>

Γ ( α , φ , R , ˆ z i )

∂ Γ ( α , φ , R , ˆ z i )

∂σ _kl =

p j=1

1 φ _j

∂ ³ b _j ( α ^T _j z ) (∂ α ^T _j z ) ³

z=ˆz i

α ^T _j(2) ∂ ˆ z i(2)

∂σ _kl α j(2) α ^T _j(2) + R ⁻¹ ∂ R

∂σ _kl R ⁻¹ ,

∂ ˆ z i(2)

∂σ _kl = −Γ ( α , φ , R , ˆ z i ) ⁻¹ R ⁻¹ ∂ R

∂σ _kl R ⁻¹ ˆ z _i(2) .

!

"

'

g( x|z )

^{@ : > 1}

!' 8G=,9 8GI9 8G69 :

ˆ z i(2)

8G=G9 8G,=9 8G,-9 8G,G9

#> !'

& (

; 8-G98-G9 >!'8G=,9 !'

˜ l( α , φ , R|x , ˆ z ) = n k=1

− 1

2

Γ ( α , φ , R )

− 1

2 log det( R )

+ p j=1

α ^T _j ˆ z i

φ _j

x ^(j) _i − α ^T _j ˆ z i

2

− 1 2

x ^(j) _i

₂

φ _j +

(2πφ _j )

− ˆ z ^T _i(2) R ⁻¹ ˆ z i(2)

2

,

^8G=+9

8 8G=-99

Γ ( α , φ , R ) = p

j=1

α _j(2) α ^T _j(2)

φ _j + R ⁻¹ .

)

Γ

ˆ z i

^F

˜ l( α , φ , R|x , ˆ z )

ˆ

z i(2)

^{1 8G=G9C}

ˆ z _i(2) = Γ ( α , φ , R ) ⁻¹ p

j=1

1

φ _j (x ^(j) _i − α _j0 ) Rα _j(2) .

^8G=H9

( 1

ˆ z i(2)

^{> :}

(

α

φ

^!'

F 7 7> ;= B

b _j (u _j ( α ^T _j ˆ z i ))

u _j ( α ^T _j ˆ z i )

ˆ z i(2)

⁷

Γ ( α , φ , R )

α _k0

l = 0

^"

l > 0

∂ Γ ( α , φ , R )

∂α _kl = 1 φ _k

e l ⊗ α ^T _k(2) + e ^T _l ⊗ α _k(2)

,

^8G=I9

x ⊗ y

^{< '}

x

y

e l

q

>

l ^th

^{= ( F}

Γ ( α , φ , R )

φ _k

σ _kl

^C

∂ Γ ( α , φ , R )

∂φ _k = − 1

φ ² _k α k(2) α ^T _k(2)

^8G=I9

∂ Γ ( α , φ , R )

∂σ _kl = −R ⁻¹ ∂ R

∂σ _kl R ⁻¹ .

^8G=I9

:

∂b _j ( α ^T _j z )

∂ α ^T _j z z=ˆz i

= u _j ( α ^T _j ˆ z i )

^8G=69