Identifying intrinsic variability in multivariate systems through linearised inverse methods

(1)

HAL Id: inria-00200113

https://hal.inria.fr/inria-00200113v2

Submitted on 20 Dec 2007

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub-

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non,

Identifying intrinsic variability in multivariate systems through linearised inverse methods

Gilles Celeux, Agnès Grimaud, Yannick Lefèbvre, Etienne de Rocquigny

To cite this version:

Gilles Celeux, Agnès Grimaud, Yannick Lefèbvre, Etienne de Rocquigny. Identifying intrinsic variability in multivariate systems through linearised inverse methods. [Research Report] RR-6400, INRIA.

2007. �inria-00200113v2�

(2)

a p p o r t

d e r e c h e r c h e

ISRNINRIA/RR--6400--FR+ENG

Thème COG

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE

Identifying intrinsic variability in multivariate systems through linearised inverse methods

Gilles Celeux — Agnès Grimaud — Yannick Lefèbvre — Étienne de Rocquigny

N° 6400

December 2007

(3)

(4)

Unité de recherche INRIA Futurs

,-./%/,0 1 2 3/45#+6,.

798;:<:;=?>@3=A:<=ABC

∗ ^DFEGHI >17KJL8<MONBP

∗ ^DQ ^N ^HH ^8;RTSVU=AW ^IAXZY ^J[= † ^{D]\(^} ^8;= ^HH =KP=`_baRAcZB8 GdH)e

†

f4gihjlknmo(prqtsvuxwzy{h[jlk[w*|}~yw

]}kyw[iA!

*i!}Azy*vkk[| gk | gikAq'k[|kj!k['A? 'q¡¢3~Ak[w

£-¤¥[¦§T¨i©¦ª« ~A{}T¬i~lx®ijnk'}]v®wzy{¯°¯wz±wy®vk[w'²|°²®!vkw{}jlk(³}j´}y{k[y{jlk?y}

ygik?®j3k[{}A®w3wz}A®i|k[wn}®i!|k{y²?y²kLwµkL|¶y{i~y{gik·|}A|°®w{²}Aw[¸¹ºy{gk®|k[zy Ayu`y{kLTyz»

jlk[Ay¼³jlk¬6}±|}Aw{vk[{kLl²-y{gi¯w¼kL½ygik²?y{wz¯|*¾T{¯i°yu3})ygik®i!|k{y²?yulwz}A®i |k[w

¯w]jl}vvk[°²°k[xuljn®°y²¾T¯Ty{k{}Ai°yulv¯wy{i®vy²}AZ¿ « ±k[u3v²Àb|®i°²yuy{ v²y{}°²°u3k|}®iv»

ykk[1Ty6y{gi¯w4wy ~k¯w¼°i±kLy{}y{gk'gi~g°²u?»Á°²jlyk[w{j3°²i~nv³}A{jTy²}A-v{kL|¶y°²ub¾T°¯i°k}A

®|k[zy b²i®vy4¾{¯°²kLw¿ « }Awwzi°k'w{}°®vy{}-°²k[w¼²-y{gik'²?yk~ Ty²}A}Zv{kL|¶y4²i³}jly{}Z½

w{®| gw¹iTy (}l}ygik]jl}A{k4k[Awz°un}!wzk[{¾T°²k4j3ky{k[w°²±k[y}(y{gik*! jlkyk w}d²?y{k[{kLwy

ygi}®i~Ag`¬6k°°»Á±xi}T¬g?uvw{|[°jl}vvk°<¿-f4gi¯w°k[wy{}9i}i²°¯wy|3?¾Ak wzk3i}i°kj·Â/f4gik

}Avk[|y{¾k1wy{}`¯vk?y{²³uÃi}i²°²yuÄiwzy{²®vy{}Z½¹y{gik·v¯wzk w{²}AÃ}'¬gi| gOw3vk[!k[vk?y

}¹ygikbw{jli°²klw{²Å[klw{|klAy{w{|3¾T¯i°²²yu9¯wTy/wzy±Ak¿lf}·°²jl²yy{}K{kLw{}i°kn°k¾k[°y{gik

x®ij!k[}(Æ³®w{®°°u9°{~Akm¼¼Ç»<y{j3kb|}w{®ijli~?È'igxuvwz¯|°j3}vvk[°F{®w(²!wz¯vk3y{gikx¾k[w{k3°»

~A}ygijw½·°²ik[/i{}Évjly{}²º9p(®w{w{¯³j3k[¬¼}A{±9{klx¾k[wzy{~Ay{kLÊ`y{gi¯w!k[[¿

Ëwzybw{²jli°kn|{²y{k[{}wkÉvgii²y{k[y}kw{®iky{gk/ik?y{²Ì!²°yu1}y{gk/jl}xik°¼Æ;¿k¿6ygik/kÉv¯w»

yk|k1®ii¯|²yu-}lwz}A°²®iy{}1y{}3y{gikx¾k[w{ki}°²k[jÈ¿]f4gik½xy{gik/w{}°®vy{}¯w|}jli®iy{k[1¾x

Í¼Î·»;yuxk°²~A}ygijw4y±?i~b{}Ìiy'}ygik3j3¯wwzi~-iybwzy{®|y{®ik/}FygiknkLwy²jy{}Ki}i°kj·¿

f4gki{kLwzk[?yTy²}AÊ|°®vkLw/Kw{}»Ï|°°k[9Í4m6ÎKÍÐ°~}A{²y{gijÑy{gyn|Äk®w{k[y{}K}T¾Ak |}jlky{gik

}Aww{²i°kn!Ty{g}°}~u·}¼wz°}T¬Ð|}Ax¾k~k[|k¬g| g`Tµdk[|¶y w'y{gikwzy!i KÍ¹ÎÒ°~}²y{gij·¿Ó*®ijlk[z»

¯|°kÉv!k[{jlk?yw}wzjn®i°¯Tyk[Ê{kL°iTy -w{kywgi~g°²~g?yygik3~}x}v9k{³}jl|kLw*}]y{gikLwzk

°²~A}ygijw½vAw4¬¼k[°²°ZAwwz}Aj3k(ik[|[®vy²}Aw¼y{}k(y±k¬gik[1®w{i~ly{gi¯w*ii}A| g¿

ÔKÕ?Öd×ØnÙ

§TÚ¥ª

Ç|k[zy AyuÎK}vvk[°²°i~½vÛÏ?y{wz¯|'Ü]i²°²yu½xÝZik[

«

ii}Év²jTy²}AZ½?Ûvk?y{²Ìi»

i°²²yu½vÍ¹ÎÍ6m6ÎKÍV°~}A{²y{gijw

∗Þ³ßàZÞâáãTäLå;äæç{èéæ;ê ëìÏåízî[ï¶îLðñ?èòóôõìæç<ôå;ö]é÷ æôøç;ù³ú[äû

†^üýã)èTà &^ý

(5)

#&% ' · 1 #6,1. 3/4

¥ ª Ç* i}ji{k1|}¯w{w?yy®vkLw3ik1w?®ik[wv®wzy{²k[°w3ik! k |}jlvykiµ»

k?yk[wjlihk[wnik-yyk3°²kLw}jn{k[®wzkLwnw{}®i |kLwn|k[zyy®vk[wlµkL|¶y?y3°k[w3|}A|°®w{²}Aw¿

!w¹|ky6zy|°²kA½°¯¾Ti²°²y*?y{²w{h?®ik'vk[w¼wz}A®i|k[w]|k[zyy®vk'k[wzy¼jl}vvk°wk¼®iik*°}

ik3{}Ai°y3j®i°²y{¾{kA¿3ÇikbvkLw(i{|²°²kLwv²À|®i°²y[ww{}®¾k?y({k[|}A?y{k3k[wzy(°¯·v¯wz}»

²i°yn°²jl²ykv³}A{jTy²}Aw*w{®i°k[w°²}Awvk/i}i°²²yLwvkLw¾Ti°²kLw*k?y{kn²|k{y²ikLw¿

Çiknw{}°®vy{}·}Awwzi°k/k[wzy*°?y~Ay{}Kvk/k!wzk[²~Aikjlk?y w4²i²k[|yw[½!|}Ajlj3kvk[w*i}i[k[w*w{®i

®vy{k[w4jlhy{k[w4°²®w|°kjlk?y*}!wzk[{¾T°²kLw4{k[°²[w®vÉ1jlhy{k[w4Ayy!®iKjl}»

ih°k*igxuvwz ?®ik*|}Aix®Z¿Fm¼k[°|}i®iy"!(®ii}i°hjlki}i°²¯wyk²x¾Ak wzk(Â°²}Aw?®ik4°¾T{¯i°y

?y{²!wzh#A®kkLwybk[Äk®Z½4°}Avk[|y{²k[wzy-¯vk[AyÌ!kb®ikK°²}AvkKi}i°²²y9v}A?yb° v¯w{!k[w{²}A

kLwy($k!i?y{klvk3°-y²°°k3vk3°L| g?y{°°²}AZ¿(}A®i°²jlykvk&%}AÊ ¯wz}Aii°²kn°²k3i}jik

i!k[°wF®njl}vvh°k4igxuvwz ?®ikÆ;wz}A®i¾k[?ykÉx~kL?ywky{k[jlwFvk|°|®i°!m¼¹Ç'È°} wvk4°'®vy{°wTy²}A

ik[w°~}A{²y{gj3kLw6²x¾k[w{k[w[½?i}®w|}w{¯$}w4iw|ky*{y{¯|°k®ik/i{}Évjly{}b°[²kw

®|Avkn~A®wwzkZ¿w®iikjl²k[ykjlw[½)i}®!w'{}A!}?wz}Aw*®i`|yhkw{²jli°k3!}A®i(~A i»

y²6°¯vk?yÌ!i°²²yv®-jl}vvh°kÆ;|(k[wzyz»)!T»Ïv{k*°kÉvwzy{k[|k'ky6°'®ii¯|²y(®iik(wz}A°²®vy²}Abv®-{}Ai°²h[jlk

x¾k[w{kLÈ¿ZÍ]w{®i²y{k½T|kyzyk¹w{}°®vy{}kLwy|[°¯|®i°kFk[®vy{°w?yvkLw°²~A}ygijlk[w)vkFyuxk]Í¹Î ½Lkj3kyz»

y ?yikniki{k/k[Ê|}Ajlvy{k°bwzy{®|y{®iknvk3v}i*k[w'j*?®?y{k[w'v®9i}°²h[j3kkLwy²jy{}Z¿

}A®i6A||+°k¹°°~}²y{gijlkÍ¼Î ½iwz}A®i¾k[?y¼°k?y[½vi}A®w¼i{}A!}?wz}Aw¼®iik¾{¯?yk'vk°njl°²°k(vk[w

°²~A}ygijlk[w)Í4m6ÎKÍ*¿ÝZkLw[w{®i°²yywZx®ij,[{ ?®ik[wwz®vkLwv}Ai[k[wwzjn®i°kLwkyk[°²°k[wjl}?y{k?y°k[w

}ik[wk{³}j|k[wvk*|kLwF°~}A{²y{gijlkLw½kyFi}®w]!k[{jlky{y{k[Ay]°¯jlw{kk-¾x¯vk|k*vk?®ik° ?®ik[w

L|®iy{}w.!3ikvk¿

/ Ù ¦L¥ × ©&0 ¥`ª ÎK}v$°¯w{y{}Ä|k{y{²y{®vkLw½¾Ti²°²yl²?y{w{h?®ik½i{}Évjly{}°²[²k½

¯vk[AyÌi°²²yA½°~}A{²y{gj3kLw6Í¹Î ky*Í6m6ÎKÍ

(6)

.%1¹ ./

«

~A{}T¬i~(x®ijnk¹}d²i®wy{¯°{¯wz±3wy®vk[wF|°²®ikwz}Ajlk4³}j }y{k[y{jlk?y]}!ygik*x®ijlk}®w

w{}®|k[wd}i®|k[zy Ay²kLwZTµdk[|y{i~ygik¹|}A|°®w{²}Aw[¿ZÛÏy{gk]kik[{~Au'w{k[|y{}L½wz®| g®|k[zy Ayu°»

uvw{k[w6k³}6wy |k'|[{k[}A®vy¼-k[x¾?}ijlk?y °!wzy{®!v²kLw'Æ}x}vbi}yk[|¶y²}AZ½?k®ik?y|}AAy{}A°;½

ky|¿È½F}nºx®|°²kLnwT³kyu`wzy{®i²kLwnx¾}°¾x²~K°{~Ak-w{|²k[?y{²Ì!|b|}Ajli®vy{i~ Æây{gkj3}»<gxuvv ®i°¯|w[½

jlkL| gi¯|w[½ik®vy{}Ai¯|w'ky|¿È¿Ëi}jl}Awzy'}]y{gi}?wzk3i°²¯|y{}w[½y'°k[Awy{y{¯°°²u·i}!i°²¯wzy{¯|

jl}vvk[°²°²~}¼ygik®i|k{y²?y{k[w(¯w/|}w{ikk[)¿bf4gik[Z½{y(³{}Aj y{gk²jl}{y?y/®i!|k{y²?yu

{}A~?Ty{}l¯w{w{®ikLw]by{gk|}?y{kÉ?y¼}}ây{k1|}jli°kÉbgi~gm¼¼Ç»<y{j3k(vk[jlvi~w|k?yÌ!|

|}jli®vy²~½d}ikn}]ygikn±Aku·ww{®ik[w'{k[~Aiwy{gik-A®!?y{²Ì!|y{}9}Fy{giklw{}®i |kLw}¹®|k[zy Ay²kLw¿

f4gk3{}Ai°²k[jÑ¯wy{}K| gi}x}Aw{kk°¯i°k3wzyy{¯wy|[°Fjl}vvk°¯w³}(ygikl²i®vy/¾Ti°k[w(wz®!| g`Aw(®i|k[z»

y igxuvwz¯|°ii}k{y{k[wF}ygik*jTyk°w]}]v®wzy{¯°{}v|kLw{w}A]y{®i °i !v}j igik}jlk

Æ¬²!)½}x}v)½vykjl!k[y{®ik½vky|¿È¿

« ±AkuÊv²Àb|®i°²yu½y{ iy²}A°°²uKk[|}A®i?y{k[{kLyygi¯w/wy ~Ak½Z¯w°²i±Ak[Êy{}·ygikgi~gi°u?»<°²jl²y{kL

wjli°i~Êv³}jTy²}Aºv²k[|y{°u ¾²°¯°²k1}º®i!|k{y²Ã²i®vy3¾T{¯i°k[w[¿ « Ãv®wzy{¯°4|w{k»

wzy{®!vu1|[·°{~Ak°ubkikÌiy*³}j y¬6}bwzy{ Ty{k[~k[w[Â

(a)²?yk~ TykkÉvk{yF®v~jlk[AyL½!w{®| gKw4°²±Ak°u

}®iw} igxuvwz¯|°¹?y{k¾T°¯w/}Anjl}kbk°¯} Tyki}i²°¯wy|v³}A{jTy²}AZ½}

(b) ?y{k[~ Ty{k

vk[|¶ylv³}jTy{}½¹wz®!| gOwliy}O}y{gkL½]jl}k-k[Awz°u }w{k¾Ti°k½Fjlky{k[w/y{gybk

°i±Ti°ky{}lygik/®i!|k{y²·¾T¯i°k}FAykk[wzy*xu1igxuvw{|[°Zjl}vvk°<¿4ÎKkygi}viw³}A

(b)^ikj

®!wzi~-}¹i}i°²¯wy|n²x¾Ak wzk/jlky{gi}viw(w{²|knygik3{kL|}T¾k[{i~}¹²!v²k[|yv³}A{jTy²}A9?¾A}°¾kLw

~Akik[°²°u/ygik*x¾k w{²}A3}igxuvwz¯|°ijl}vvk°<¿F}®i~Agi°u3w{!kL±xi~½y{gi¯w¹x¾k[w{²}A3y{ wz³}jwy{gik

v³}A{jTy²}A·²?y{}bb¾?{y{®°w{jli°²k}y{gik/¾T{¯i°k(}FAykk[wzy[½!!k³}k/ii°ux²~ly{}b²y'wzy!i

wzyy{¯wy|[°k[wzy{jTy{}¿f4gi¯w'Ìk°¯K¯w(| ±xi}T¬°k[v~Ak[1y{}g¾k A®yk3-°{~Ak/²!v®wzy{°}yk?y{¯°

Kygik|}?y{kÉ?y'}] i¯·~}T¬4y{g·}]v®!wy{¯°j3}Ai²y{}i~-!KiTy -|?®iw{²y{}9w{uxwzy{k[jw¿'o'ik

}4y{gikb²~9¯wwz®ikLw iA|¶y|kw/y{}9°²jl²y/y{}K{kLw{}i°k3°k¾Ak°Fygikbx®ij!k[n}/Æ®wz®!°°²u°{~Ak

m¼¼Ç»<y{j3k/|}w{®ijli~?È¼igxuvwz¯|°)jl}xik°Z®iw4w{vkygik²x¾Ak wzk(°~}A{²y{gijw[¿

f4gi¯wkn|}|k?y{ Tyk[w}AÄy{gikwz²y{®!Ty{} ¬gkky{gik[{k-w

irreducible ®i|k{y?yu}A

¾T{¯i°yu ÃygikKii®vybjlky{k[w}(`igxuxw{¯|°4jl}vvk°<¿ ÛÏx¾k w{k-i}i°²¯wy|-y{kL| gii ?®ik[w

ÄiTy Aw{w{j3°¯Ty{}Ê}j3ky{k['¯vk?yÌ!|[Ty²}A`{k3i}yik[¬ Æk¿~¿ ¼kL| ±"!#x l}AfAy}°¯

!$#&%x È¶¿-ÛÁy/jlui}y!k®i?y²°"A®ykb{kL|k[Ay°²uÊy{gy³®i°°¹i}i°²¯wy|lx¾k[w{²}A¬4w|}w{¯vkk[)Â

f4gk(iwzy{²®vy{}}²?y{wz¯|3Æ³}{kLv®|i°k½v°k[y{}A{uiÈFii®iy*®i|k[zy ?yuww{k[| gikL)¿]m¼°Aw{w{»

|[°TyAw{w{j3°¯Ty{}} jlkyk¯vk?yÌ!|[Ty²}A1yk[| gi?®ikLw²x¾A}°¾ky{giknk[wzy{jly{}·}ii®vy

! jlkyk w'Æ}4²y°Z|}!vy²}Aw[½x³}4kÉijli°k²jlky{k[}}°}~AuvÈFy{gy*k'®i±?}T¬-i®vy*g?uvw{»

|[°°²ulÌÉxkL)¸xy{gw6¯w4Ty®i°²°u|[|}jlik[xubkLwy²jTy²}A-®i!|k{y²?yu3³}4¬gi¯| g-¾T{¯|k'ju

k|}Aj3®vy{kL)¿('}T¬¼k[¾k[w{®| gkLwy²jy{}`®i|k[zy ?yug!ikwy}·!kli®ik°u9k[iwzy{k[jl|3}

k[i®|i°k½x²-y{gikw{kw{k'y{gy6²y¬²°°)vk[|{kLw{k¬²y{gbygik(²k[|¶y²}A1}°¯~k¼}w{k¾TTy²}A-w{jli°²kLw¿

f4gwwi}y3wTy{¯wz|¶y}u ygik|[w{k[wn¬gi| gÃj3}y{¾TTy{kL ygik-ik[w{k?y3{kLwzkL | gZ½¬gkkxu)!®i

jlkL| gi¯|°w{uvwykjw4v}3k¾xik|k?y{²!wz¯|°²°ul¾{¯°²k²i®vy[¿FÛÏi®vy¾T°®ik[w4v}li}y6}Ai°²ubw{®vµdk

³}j °¯| ±l}±?}T¬°²kLv~kA½Ai®iy¼ygikub°w{}nigxuvw{|[°°²u3¾Tu³}j }AikkÉv!k[{k|k*y}3i}ygik6²-y{gik

|[w{k[w}*}T¬ }w{k¾TTy{}!wy ±Ak Ty¾T²}A®wy{jlk[w}¹ygikbukL(}Ä+®|y{®y{i~K²¾Ak!kL)½Z}

y®ii®i°k?y,®i¯}T¬ jlk[Awz®{k[j3k[?ywF-x®|°²kL]k[A|¶y{}AF¬yg-| gi~i~gikLTy¹kÉv| g!i~Aki{}A!k[z»

y²kLw¿FÛÏygi}Aw{k/|AwzkLw½?y{gk/wzÅk}}!wzk[{¾TTy²}Awjli°k[ww{gi}A®i°1j3!|¶yygikk[wzy{jTy²}A·A||®iA|uA½

®vy*yw{gi}A®i°1i}y*{kLv®|kygik¾T|k}y{gikg?uvw{|[°®|y{®y{}w[¿

(7)

Î9y{gik[jTy{¯|°²°u½x}Awzk[{¾Ty{}w¼kjl}vvk°°²kLb¬²y{gn¾Ak[|¶y}¼}igxuvw{|[°¾{¯°²kLw

y y{g!Tyk

|}ik[|¶yk[ly}n®i!|k{y²-²i®vyw

xygi}®i~Agbnvky{kjl²wzy{¯|Æ;-wz®i!}?wzkLv°²u ?®i²y{k'¬¼k[°²°²»<±xi}T¬È

g?uvw{|[°j3}vvk[°

y=H(x, d)^¿ ^« w'1|°k[(v²µk[{k[|kny{}|°¯wwz¯|°j3ky{k[¯vk?y{²Ì!|[Ty{}

x ^¯w

}yw{®ii}Aw{k[-y{}g¾Ak3ÌiÉvkL)½°k²y*®ii±xi}T¬igxuxw{¯|°)¾T°²®ikAÂFÛÁy*¬²°°Zk/jl}vvk°°²kL· !v}j

¾T{¯i°k[wy±x²i~·v²µk[{k[?y(k[°²¯wTy{}!w*³}AkL| g}w{k¾TTy{}!w¿/f4gikli®{}Aw{k}¹ygik°~}²y{gij

¬°°F!kly}1kLwy²jTykly wi}i²°²yu9vwzy{i®vy{}³®i|y{}`²!wyk[}6y w(!}A²?y/¾T°®ikA¿o'y{gik

}y{gik[lgZ½

d wzyiw3³}AÌÉxkL ²i®vyw3ygTybju kik[w{k?yKÆ³âÈ3¾{¯°²kLw®i!vkl³®i°°|}?y{}°

Æk¿~¿kÉv!k[{jlk?y°¼|}v²y{}!wÈ½F}1Æ²âÈn®i|k[zy ?y{k[w3Tµdk[|¶y²~Êwz}Ajlk-jl}vvk°6²i®vywygTylk

|}w{¯vkk[by{}kik~A°²~i°k}A4}FwzkL|}!iu²jl}{y|kA¿

ÎK}A{kw{!kL|²Ì!|°²°uygik(³}°°²}T¬i~3jl}xik°Z¯w*|}Aw{vk[{kL

Yi=H(Xi, di) +Ui, 1≤i≤n ^Æ ^!LÈ

¬gik[{k

(Yi) R^p vk[i}yk[w6y{gk/iTy ¾Ak[|y{} w[½ H vki}y{kLw±xi}T¬1³®i|y{}³}j

R^q ^y} R^p^½

(Xi) ^² R^q¹ vk[i}yk[w4i}A1}Awzk[{¾Ak[1v}Aj iTy i½iww{®ijlk[1²ikkvk[?y'1¯vk?y{¯|°²°u

v¯wy{i®vyk[ÄÆ³<¿<¿)¿Èl1¬yg9vwzy{i®vy{}

N(m, C)^½

(di)iki}y{k[wÌiÉvk[1}w{k¾kLb¾Ti°k[w[½?¬²y{gKvjlkw{}

q2

½

(Ui) vk}y{kLwjlkLw{®i{k[jlk?yz»Ájl}xik°k{}Aw[½!Aw{w{®ijlk[Ê;¿;¿)¿(¬²y{gÄvwzy{i®vy{}

N(0, R)^½ R

!k[²~±?}T¬Z¿Ü]{¯°²kLw

(Xi)^! (Ui)k(Aw{w{®ijlk[-y{}lk²!vkkik?y[¿

f4gikj ¯wy}k[wzy{jly{kygik/jlky{k[

θ= (m, C)^¿ 6k³}kk?y{k[{i~ly{gky{kL| gii¯|°{y*}

ygi¯wi}i°kj·½i³k¬ {k[j±xw4{k'y{}kgi~gi°²~Ag?y{k[Z¿

1Õ º¨v§! #"%$ & (' )*+ $ , ( -&

.0/ 01, (' 2 $ 3 + &' ( 4'5' 2 3 6

Ui

7'8' 9: $ <; =' >? @( & >

(,A 8 $&B.?/ (' <;, ( 3"C

Ui

*3 =' DE: $ 7 $&* ;- @

$& F 8; &$ G & H ) JI0K .

L @7;

R ^M( ^$ ^+N.

1Õ º¨v§PORQ * *) *; S' 4 $ T: & U " +<;

S' 9 V

"

* XWCYZ8Z<[C\; ]8 V ^'

3*+A_ & 3./ (+ `Wba*UcedG;f )g ;

h )g ;%i( KjA1k ; =.\; (' C

X ^Fl$ ^$ ^H'V ^Bl n

1, ;* nm)Ko U Wqp,rr8s\t " H' 88+ 3 j

$( * F 1 U:C0: $ 7; 3+A

.

(8)

1Õ º¨v§ " U $ T:_ & & 3+ 8+( $

3D @ (' 7 DM * V.

sxk[¾k[°v²À|®i°²y{k[w1jlu !kLL¿ o'ik9¯w{w{®ikÊ¯wygikÊ¯vk[AyÌi°²²yu }(y{gkÊj3ky{k[w}

ygikv}j¾T¯i°k[w

(Xi)½]¬gi¯| gO|[Ãk}®vyl}k[A| g y{gikK¾T°i°kb²i³}jly{}º¯w3i}y

¯| g k[i}®~gÆâ³}A-kÉij3°²kKk[|[®w{kKy{gk9i} }Awzk[{¾Ti°²kK¾T{¯i°k[wi²jlk!wz}V¯wb°¯~k[ly{g

ygiklv²jlk[wz}}]ygik3}w{k¾kL·}®vyi®vyw È¶¿ « i}ygik(¯w{w{®ik3wy{gky²jlk3ikkLvk[9y{}·|}Aj3®vy{ky{gik

g?uvw{|[°³®i|y{}

H^½iw{²|k H ¯w4}ây{k[-ygik({kLwz®°y4}|}jli°kÉ1|}xik¿ « !}?w{w{²°²k!wz¬6kL½A®!wzkL

`y{gikl³}A°²°}T¬i~½Z¯w(y}°ik[{¯wzk3ygikjl}xik°¹{}A®i`-ÌÉxkL`¾°²®k

x0

¿bf4gkb|}!wz¯vkk[jl}vvk°

k[|}jlk[w

Yi=H(x0, di) +JH(x0, di)(Xi−x0) +Ui, 1≤i≤n ^Æ<È

¬gkk

JH(x0, di)¯w4y{gik?|}i¯1jTy{²É-}y{gik(³®i!|¶y{}

H x0

½i¬²y{gKvjlkw{}

p×q1

¿

f4gi¯w¼°²ik[{¯wTy{}ljlkygi}vbgAw¹w{}jlkv ¬A| ±xwÆ³³}6kÉijli°k*gi}T¬ y}| gi}x}Aw{kygik°²k[¯w{»

y²}A9}AyL½ii}Év²jTy²}A·k{}Aw[½iky |¿øÈKi®iy'²yv}xk[w'i}y'?¾A}°¾k/|}jli®vy Ty{}!°}Ax®ijlk¯|°

iÀb|®°y²kLw/Æ³}y{gik[4y{g1ygik/i²y{¯°|}jli®vy Ty{}}y{gkAA|}-jTy{²ÉÈ41~¾k(k[w{}!i°k

k[w{®i°²yw4·jlxu-|AwzkLw¿

f4gikliy

(Xi) ki~1}Ê}Aw{k¾k[Z½ygiklk[wzy{jly{}9{}Ai°²k[j w(1jlwwzi~iTy 1wzy{®|¶y®ik

{}Ai°²k[j y{gy4|[bkw{}°¾kLl¬ygbÍ¹Î·»yuxk°²~A}ygij·¿f4gik'Í¹Î °~}²y{gij ¬6Aw¹vk[¾k[°²}A!kL

xu`k[j3!wyk *B.ÃÆ!$#? È(Ä¯w/ yk Ty{¾kb°²~A}ygij ¬²y{gÄy¬6}Êwzy{k[wÂÍ s?yk Æâ³}AnÍFÉx»

k[|yy{}!È!ÊÎ+s?y{k[ Æ³³}Î9TÉvjlwTy²}A!È¶¿ « kÉxy{k!wz}Êy{}y{gi¯w(°~}A{²y{gij·½Z|[°°²kL9Í6m6ÎKÍ

°²~A}ygij·½³}A4ÍFÉv!kL|¶yy{}v»zm¼}!vy²}A°dÎ9ÉxjlwTy²}AbÍ]²y{gik[[½v¬4w6i}}Aw{k[b?ubÝ²®1®v»

² Æ!# #xÈ¶¿(f4giw(°²~A}ygij ¬g| g`|Ê!k3kÉvk[|¶yk[9y{}·|}x¾Ak~kjl}kn i¯v°uy{gÊÍ¹ÎÒgw

°w{}9kk[Ãkjli°}TukL y{gikik[w{k?y3wzy{®!vu¿ Ëi}Ajl}vvk[°Æ<È¶½]y{gikÍ6m6ÎKÍ °~}²y{gijgAwnkk[

{}A!}?wzkLbvkk!vk?y{°u-xukm¼k[|u9Æ !# #?È¹®vk4ygikj3kLm¼² |kÎKky{gi}v¿

f4gik1kwn}~Aiw{k[ÄAw(³}°°²}T¬*w[¿KÛÏÃsxkL|¶y{} v½Ê|²y{k}Äwni}}Aw{k[y{}kw{®iky{gik

¯vk[AyÌi°²²yu·}]ygik3°²ik['jl}vvk[°6Æ<È¶¿(ÛÏÄsxk[|y{}Ê¢½!ygik3Í¹ÎÒ99Í6m6ÎKÍr°~}A{²y{gijw'k

ik[w|²k[9K|}Ajlk[y}-k[wzy{jTyk/y{gk°²k['jl}xik°Ty'g!)¿'ÛÏ`sxkL|¶y²}A9½!y{gikny¬¼}°~}»

²y{gijw{k/ii°²kLy{}y¬6}kÉij3°²kLwÂ6f4gikÌ wy*kÉijli°kn|}Awz¯wzyw}Fw{jn®i°¯Tyk[Kiyl}iyikL

³}j -?®i²y{kwzjli°ÌkL !}?}vvi~3jl}xik°<¸xy{gikw{k[|}AbkÉijli°k'¯wnk[°»Ï|Awzk*y{gik[{jl}»Ágxuvv®i°|[w

wzy{®!vu-|{k[1}®vy ¹°²kL|¶y{¯|²y/'k/Ëi |klÆÍ¹ËÈ¿ « ikFvw|®!w{w{²}A-k[iw4ygikkL¿

·· 0./O,#+.

ÛÏy{gw/w{k[|y{}`|{²y{k[{}w({}A!}?wzkLKy{}·k[wz®{k3y{gk3¯vk[AyÌi°²²yu9}¼ygikl°²ik[(j3}vvk[°Æ;AÈ¶½

k¬²yzy{k[1gkkKlwzjli°²²ÌkL-³}A{j·Â

Yi=HiXi+Ui, ^³} i= 1, ..., n ^Æ;¢AÈ

¬²y{g

Yi

}w{k¾kLb¾kL|¶y}4²

R^p

½

(9)

Hi

3±?}T¬1jTy{²É-¬²y{gKvjlkw{²}A

p×q^½

Xi

· v}Aj ¾Ak[|¶y}/¬²y{gºi²jlk!wz}

q^½y{gk Xi

wn{kbAw{w{®ijlk[`<¿<¿)¿K¬ygºv¯wy{i®vy²}A

N(µ,Σ)^½ (µ,Σ)!k[²~®ii±xi}T¬Z¿

Ui

b¾k[|y{}A¬yg9v²jlk[wz}

p½!!y{gik

Ui

w*{k/Aw{w{®ijlk[1y}b!kn;¿;¿)¿6k{}Aw6¬²y{g9v¯wzy{»

i®vy²}A

N(0, R)^½ R !k[²~±?}T¬Z¿

f4gik[Z½

Yi

w(·p(®wwz¯Ê¾k[|y{}A'¬²y{g

E(Yi) =Hiµ ÊÜ]

(Yi) =HiΣH_i^T +R¿nËi®i{y{gk

}yy{}w4?y{}vv®|kL)Â

Y =





 Y1

¿¿¿

Yn







½

H=





 H1

¿¿¿

Hn







½

Γ=





 H1

¿ ¿ ¿

Hn







½

Σ=





 Σ

¿¿ ¿

Σ







R=





 R

¿ ¿ ¿

R







¿

'k|k½

Y gAwi²jlk!wz}

np^½¬gi°k H^½ Γ^½ Σ R{kFjTy{¯|kLwZ¬²y{g/k[w{k[|¶y²¾Ak¹vjlkw{²}Aw

np×q^½ np×nq^½ nq×nq np×np¿]f4gx®w[½

Y wbp(®wwz¯1¾k[|y{}A¬yg

E(Y) =Hµ

Ü]

(Y) = ΓΣΓ^T +R^¿

f4gikjl}vvk[°Zww¯-y{}kvk[?y{²Ì!i°k(F!1}Ai°ub³}*°²°

(µ1, µ2) (Σ1,Σ2) Hi(µ1−µ2) = 0∀i

Hi(Σ1−Σ2)H_i^T = 0∀i ⇒

µ1=µ2

Σ1= Σ2,

}A4k?®i¾°²k[?y{°u-²-}Ai°ub³}*°²°

(µ1, µ2) (Σ1,Σ2) H(µ1−µ2) = 0

Γ(Σ1−Σ2)Γ^T = 0 ⇒

µ1=µ2

Σ1= Σ2.

§ Ù¹Ù ¥¦ Ù Q (,

q≤np;4m)$&kW=f7\ &1 8 +K E:

(H) =q^.

* 6 Â

Ë² wyL½i°kyww{®ijlk'ygTy i±)Æ

H) =q^¿

!A¿6ÛÏygiw|w{k½

H ¯w4²TkL|¶y²¾Ak/

H(µ1−µ2) = 0⇒µ1=µ2

¿

i¿6ÛÁy/¯wi}T¬ i}T¾k[Êy{g!Ty

Γ ¯w(³®i°°]i±

⇔ Γ^TΓ ¯w?¾Ak wzi°k¿-sx|k

Γ^TΓ ^¯w/ q×q

jy{É)½

Γ^TΓ¯w]x¾k[w{²°²k²l}Ai°un²

Γ^TΓw¹²TkL|¶y²¾Ak¿]sx}½?°ky

v ^{k (k}(Γ^TΓ)^¿

f4gk

Γ^TΓv = 0 ⇒ v^TΓ^TΓv = 0 ⇒ ||Γv|| = 0 ⇒ Γv = 0 ⇒ v = 0 ^wz|k Γ ^¯w

TkL|¶y{¾kA¿,'*k[|kA½(k

(Γ^TΓ) ={0}1yg?®!w

Γ^TΓw4x¾k[w{²°²kA¿

m¼}Ax¾k w{k°u½Z°kylwwz®ijlklygTy

Γ^TΓ ¯w/x¾k[w{²°²kA¿1ÝZky

v ^k-(k(Γ)^¿ Γv = 0 ⇒

Γ^TΓv= 0⇒v= 0^wz|k Γ^TΓ¯w]TkL|¶y{¾kA¿'*k!|k(k[

(Γ) ={0} Γ¯wF³®°²°i±d¿

(10)

¢¿ 'k|k

Γ(Σ1−Σ2)Γ^T = 0⇒Γ^TΓ(Σ1−Σ2)Γ^TΓ = 0^¿sx|k Γ^TΓ w?¾Ak wzi°k½y

jli°²kLw4y{gy

Σ1−Σ2= 0^¿

m¼}x¾k[w{k°u½²iy{gk6jl}vvk[°?¯w¯vk?y{²Ì!i°²kA½³}F°°

(µ1, µ2)^½ H(µ1−µ2) = 0²jli°k[w

µ1=µ2

¿

'*k[|k (k[

(H) ={0}i±)Æ

HÈ¼¯w4k?®°dy}

q^¿

f4gi¯w'i{}A!}?wz²y{}·i}T¾xik[w*Kk[w{u·|yk²}A1y}1| gikL| ±1yy{gkn¾k[{u1!k[~iii~b}Fy{gik3wy®vu

²¹ygik3²x¾k[w{kn{}Ai°²k[jÑTy(g!9gAww{}°®vy{}Z¿¹A|¶y{¯|°²°u½)²]ygiklj3}vvk[°w(i}y¯vk[AyÌi°k½

w{k¾Ak °wz}A°²®vy²}Aw4|-kk[x¾?¯w~kL)½x°°!w{k[b}Abygik'®!wzk(}kÉv!k[zy4}4k[i~ikk[{i~®v~k[jlk?y[¿

ÛÁkÉvk{y*±xi}T¬°²kLv~k(¯w| gki}A®i~g½Ìwzyw{}°®vy{}

y{gy*¬²°°k°²°®wzy{ Ty{kL²y{gk }x}xi²i~

kÉijli°k

|}!wz¯wy w}4ÌiÉvi~9w{}jlk}6y{gik- jlkyk w(}4y{gkbi}i²°²yuÊv¯wy{i®vy²}A }4y{gik

!v}j ¾Ti°²kLw

X¸]y{gi¯wnjuÄkby{kLv}®w[½Fk[w{!kL|¯°°²uÄ¬gik[ y{gk1jlky{k[y{}!k1ÌiÉvk[Ãk wzyivk¾x¯Ty{}!w½Z~Akik[°²°ujl}kv²À|®i°²yy{}·k¾T°®y{k-{}y{gjlkL`¾°²®k[w[¿ «

w{j3°²k[3ii}A| g

²°°²®!wyy{k[º²Ãy{gik1{kL°²»Á|[w{kly{gik[{jl}»Ágxuvv®i°|[w/kÉij3°²k

wy}ww{®ijlk

w{}jlkl}4ygik

Xw/{k3ÌiÉvk[ Æk?®°]y{}9·jlk[`¾T°²®ikbiky{k[{jlik[ÄxukÉv!k[zy wÈ½Zyg?®!w/ik[~°k[|¶y²~

ygiknwzy{}v| gAwy|3y{®ik¿+ ¼®iynw¬6kl¬²°°¼w{kkA½Zy{gwwzjli°²²Ì!|[Ty{}ÄgAw·i|kÂ/ÛÁyn|[Ä°²kLy{}

}T¾Akk[wzy{jTy{kwy 1vk[¾xy{}w6³}A4y{gikkjii~3 jlkyk w¿

1Õ º¨v§

W3d*H' 3*/&1 8+(\ /' 8 3

q≤np ^*^5'*j

Y*; ( 7 7 $" +8U V.^,

q=p= 1 ⁰ np= 2 ⁷ ^M8 m

C^.+ np= 1 ^<; ^A ^{^} ^_ ^8.L ^@ ^;

F' 3 +A; 7'8'V 7n *En & ) 7

3. T '@

n0q≤np^; ^$ n0

`p,.

/

/n. ¶,#+

ÛÏ`ygik-|}A?y{kÉxy}jl¯w{w{²~·Ty½`kLwy²jy{}Äjlky{g}xÄ°¯~k[°²u9A| ±xi}T¬°²kLv~kLÊAw{k[°²k[¾T?yw

ygiknÍ¼Î °~}A{²y{gjÆ;k[jlwyk B.½!#x È¶½¬gi| gÊw9²y{k[y{¾k3°~}²y{gij ¬yg9y¬6}-wzy{k!wÂ

Í sxy{kOÆ³ÍFÉvk[|¶y Ty²}A!È!KÎ5s?y{k[ºÆÎ9ÉxjlwTy²}A!È¶¿ « KkÉxy{k[wz}Kvk[¾}y{k[y}b|[|k°k Tyk(y{gik

Í¼Î °~}²y{gij·½¬gi¯| g¯wF±xi}T¬3y{}/}ây{kk[|}A®i?y{k[¹wz°}T¬O|}x¾k[{~Ak|k6w{²y{®y{}w[½¯wFygik*Í6m6ÎKÍ

ÆÍFÉv!kL|¶y Ty{}i»m¼}Av²y{}°ZÎ9TÉvj3¯wTy{}1Í]²y{gik[ È°²~A}ygij }FÝZ®··®i² Æ !$#&#?È¶¿

ÛÏy{gw!kL½xy{gik(³}v|®!ww4°A|k[1}ygik°²ikLw{k[-j3}vvk[°¹Æ;AÈ

Yi=H(x0, di) +JH(x0, di)(Xi−x0) +Ui, 1≤i≤n

¬gik[{k

JH(x0, di)¯w6y{gik?|}i¯1jly{É1}

H x0

½i¬ygKv²jlk[wz}

p×q1

¿

f4gik(³}A°²°}T¬i~}yy{}w4®!wzkL)Â

Zi= (Y_i^T, X_i^T)^T^½ i= 1, . . . , n ¯wvk[i}y²i~3ygik/|}Aj3°²ky{k[

Ty¿`k¼vk[i}yk

hi=H(x0, di)^¸ Ji=JH(x0, di)!/|}Awzk#?®ik?y{°u

Yi=hi+Ji(Xi−x0)+Ui

¿

Î9}k}T¾k[[½?¬6k(iki}y{k

Ai=Yi−hi−Ji(m−x0) ^¸ Bi=CJ_i^T^¸ Vi=JiCJ_i^T +R^¿

sx}Ajlknik°²jlu{kLwz®i°²yw'k®w{k³®i°y{}1vkÌk/y{gknÍ¹ÎÒ·Í6m6Î9Í °~}A{²y{gjlw*³}ÎK}vvk°

Æ<È¿

(11)

f4gik(¾Ti°²k

Yi

w3°²ikL|}Ajniy{}-}p(®w{w{¯-¾T{¯i°k[w6¬²y{g·jlk[1!-¾T¯|k

Í (Yi) =hi+Ji(m−x0), ^Ü](Yi) =JiCJ^T_i +R. ^Æ?È

Zi

¯w*p(®ww{1¾Ak[|¶y}4¬²y{g

Í (Zi) =

hi+Ji(m−x0) m

1Ü]

(Zi) =

JiCJ_i^T +R JiC CJ_i^T C

. ^Æ^È

Xi|Yi

w1°¯wz}ººp(®w{w{¯ ¾kL|¶y{}A9Æ³®w{i~ºÄy{gk}kjÒ}V|}Av²y{}°v¯wy{i®vy²}Awb}(

p(®ww{1¾Ak[|¶y}L½vwzk[k'³}A²!wy |knsv}{y½!$#&#AnZ¿,% %AÈ¼¬²y{g

Í (Xi|Yi=yi) = m+CJ^T_i (JiCJ^T_i +R)⁻¹(yi−hi−Ji(m−x0))

= m+BiV_i⁻¹Ai

ÆAÈ

Ü]

(Xi|Yi=yi) = C−CJ_i^T(JiCJ_i^T +R)⁻¹JiC

= C−BiV_i⁻¹B_i^T. ^Æ< È

« y*²y{k[y{}

k+ 1½xygikÍ¹Î°~}A{²y{gj w|}jl}Aw{k[1}y¬6}wykw[Â

f4gik·Í s?y{k[rÆÍFÉv!kL|¶y Ty{}È¶Â·ÛÁy-|}Awz¯wzyw3}(|}jli®iy{i~

Q(θ, θ^(k)) = E[L(θ, Z)|Y, θ^(k)]

¬gikk

L w6ygik/|}Ajli°²ky{kL-°}~A°²±k[°²gi}x}v)¿

f4gik/Î&s?y{k[ ÆÎ9ÉxjlwTy²}A!È¶Â

θ^(k+1) ¯w4}iyikL1xubjÉxjlw{i~3y{gik(³®i!|¶y{}

Q^½ θ^(k+1)= arg max

θ∈ΘQ(θ, θ^(k))^¿

f4gik³}°°²}T¬i~/k#?®Ty²}Aw¹vkLw{|{kjl}k*ik[|w{k°unygikÍ¼Î °~}A{²y{gj y{gkzy|®i°¼|[w{k

}jl}vvk°¼Æ;È¿]Ëi®i{y{gki}yTy²}A1¯w®w{k[ZÂ

A^(k)_i =Yi−hi−Ji(m^(k)−x0), B_i^(k)=C^(k)Ji^T

V_i^(k)=JiC^(k)Ji^T +R.

¬gik[{k

m^(k) ^! C^(k) {k'y{gk¾°²®k}

m C yyk Ty{}

k}ygikÍ¹Î&°²~A}ygij·¿

f4gik[Z½Z!°uAy|[°k?®y{}w|Êklik²¾Ak[·y}|}jli®vyk

m^(k+1) C^(k+1) ¬²y{gÊk[w{k[|¶y y} m^(k) C^(k)^¿

§

Ù¹Ù

¥¦

Ù O "

H'& > ) 4'

(m, C) ^60m) ^Wqp\ ^?m ^q'

6 Ud0m *

m^(k+1)=m^(k)+1 n

n

X

i=1

B_i^(k)(V_i^(k))⁻¹A^(k)_i