Subjective Bayesian statistics: agreement between prior and data

HAL Id: inria-00071367

https://hal.inria.fr/inria-00071367

Submitted on 23 May 2006

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub-

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non,

Subjective Bayesian statistics: agreement between prior and data

Nicolas Bousquet

To cite this version:

Nicolas Bousquet. Subjective Bayesian statistics: agreement between prior and data. [Research Report] RR-5900, INRIA. 2006. �inria-00071367�

ISRN INRIA/RR--5900--FR+ENG

a p p o r t

d e r e c h e r c h e

Thème COG

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE

Subjective Bayesian statistics: agreement between prior and data

Nicolas Bousquet

N° 5900

Mai 2006

Unité de recherche INRIA Futurs

/2.2/

346587:9<;>=?@7:A0=CBA0D8E

∗

FHGJILKNMO PQSRUTWVYX[Z\ICKNMCX&]L^`_`acbdZebdfgX

hie^kjlMLZmTYnponOF

q&rkscs>^8i[Z&tWM#ieMC]uGJMCie]uGcM a,vxwky8z`z*R|{}r`b~kz8z`#R€~`@scrk_8MCX

‚„ƒ$…C†L‡‰ˆJŠ`†`‹Œ GJMLaŽHrV`M‘X[b’rkab“aWf”MLieMLac]LMNb“Xi\M‘•8–cb—ieMCt˜Z\^™M‘XlZeb—Kr‰ZeMNZ\GJMs>rkiurkKNMZeMLi#^kf r™tWM‘]b’X[b“^`aWš

Kr`›œb“aJ_KN^WtWML6ž>Z\GJMsci\b“^`i&KN^WtWMC—“b—ac_ŸK–cX[Zm ,MX\s>M‘]b—¡cMCt£¢ b—Z\G˜_`ieMCr‰Z]CrkieM`žJM‘X[s,MC]Lb“r`—“V/¢ GJMLa}X\–J Wš

jlM‘]¤Zeb—¥8M›YaJ^‰¢ “MCtW_8MŸb’X(–>X[M‘t§¦¨©actWMCMCt§žpŽHrV8MCX\b“r`ab—aWf”MCi\MCac]M/b’XjlMC^`scr`ietJb—ªCMCt¢ GJMCa«ZeGJMLieMb’XNr™Ze^œ^

’rkie_`MtJb“Xe]ieMLscr`ac]V„ >MLZl¢MCMLa™sJieb—^8i&›YaJ^‰¢ “MCtW_8Mrkact£^8 cX[MCi\¥8MCt£tJrkZerJ¦ ¬ci\^8K­r@ZeGJML^8i\MLZ\b’]Lrks>^8b—aœZ^`f

¥Yb“ML¢#žsci\b“^`iNrkact®“b—›8ML“b—GJ^Y^Wt«]L^`acX[Z\b—Z\–WZeM/Z\GJM™]^8KNsJ—MLZ\M£ŽHrV`MCX\b’rkaKN^WtWML&r`act«b—ZŸX[MCMLKX@i\MC—MC¥r`aœZ

Ze^/X[Z\–>tWV™Z\GcMŸrk_`ieMLMCKNMLaœZm^`fpZ\GcMNtcr‰Zer„Z\^/ZeGJb“XKN^WtWMC¯¦@¨©a˜ZeGJM@fgr`]ZCž§ZeGJMtWMZeMC]¤Zeb—^8a*^`frs>^œX\X\b“ J—M

]L^`aW°>b“]Zm ,MZl¢MLMLa}Z\GJM@sJieb—^8imrka>t£Z\GJM@^` >X[MCi\¥8MCt/—b“›`MC—b“GJ^Y^Wt™ieMLKrkb“acX±rŸX[b“_`acbd¡>]CrkaœZmsci\MC—b“KNb—acr`i\VZ\^

X\–J WjlM‘]¤Zeb—¥8M„ŽHrV`MCX\b’rkab—aWf”MCi\MCac]M8ž0scrki\Z\b’]–J’rkie“V}b—a²b—actJ–cXlZei\b’rki\MC—b’rk cb—“bdZlV8ž$r`actX\MLMLKXaJ^kZZe^}Gcr¥8M

 ,MLMCa™K–c]uG}X[Z\–>tWb—M‘t™ ,Mf”^`ieM`¦H¨©a£ZeGJMsJieMCX\MLaœZieMLs,^`i\ZCžJ¢MsJie^`s,^8X\MZ\^„–cX[M(r]iebdZeMLieb—^8a

Coh(π;Xⁿ)

KNM‘r`X\–Ji\b“aJ_Z\GJMr`_`ieMLMLKNMCa8Z^kfZ\GJM#r¥‰rkb““r` J“MmsJieb“^`i

π r`actZeGJMtJr‰Zur

Xⁿ b“aWf”^8i\Kr‰Zeb—^8a„¢ b—Z\G™r(ierkZ\b“^

^`f³–J“— cr8]u›œš<oMLb“ J—MCitWb’XlZurkac]LMCX >MLZl¢MCMLaZeGJMsci\^8s>^œX[M‘t´sJi\b“^`i(rkact ,MLa>]uGJKrkie›*¢MCr`›œ“V^`iac^`aWš

b“aWf”^8i\Kr‰Zeb—¥8M(sci\b“^`i#tWb’XlZei\b“ J–WZeb—^8acXC¦@¨<f

Coh(π;Xⁿ) ≤1ZeGJMLa´ ,^kZ\G´b“aWf”^`ieKr‰Zeb—^8acXrkieM@tWM‘]’rkieMCt*b—a r`_`ieMLMCK@MCaœZC¦F^#–cX\Mb—Z¢MG>r¥`MZ\^#tWML¡caJM&r]’r`XeX0^kf,K@b“aJb“Krk“Vb“aWf”^`ieKr‰Z\b“¥`MsJie^`s,MLixsJieb“^`ixtWb’XlZei\b“ J–Wš

Zeb—^8acX f”ie^`KµtJrkZerc¦FHGJMLieMf”^8i\M#¢Msci\^8s>^œX[Mr„_`MLacMLiurkr`sJsJie^8r`]uG£–cX\b“aJ_}¶@·¹¸c·¹¶º‰»0¼¯½eº‰·¹¸c·¹¸Y¾¿Lº‰¶À,»—Á¿

r`act«ÀJ‰¿u¼©ÁL½¤·g‰½NÀ:½¤·g‰½u¿¤žH^‰¥`MLiu]^8KNb—aJ_´X\^`KNM™tWbdß]L–JdZeb—M‘XNZ\^KN^WtWMC&b—_8aJ^`iurka>]M`¦ÄFHGJb’XŸrkscsJi\^œr`]uG®b“X

r`sJsJ“b—M‘t(Z\^#ZeGJM&MÅWs,^`aJMCaœZ\b’rkcr`act Œ MCb— J–c—,KN^YtJML’XLž`Z\GJM&KN^œXlZp–>X[M‘t@tJb“X[Z\ieb— c–WZ\b“^`acXpb—aN“bdf”MLZ\b“KNMsJie^` Wš

“MLKXC¦m¬$b“acrk“—V£¢MtWb’X\]L–cXeX&ZeGJM–cX\M^`fxZ\GJM

Coh ]ieb—Z\MLieb“^`a}r`XmrZe^œ^8^`fp]Lrk“b“ JierkZ\b“^`a£f”^8imX\–J WjlMC]Z\b“¥`M sci\b“^`iHKN^WtWML“—b“aJ_>¦

ƙǜÈ:ÉeÊË ‡‰Ì$…`‹ ¨©actW–cX[Z\ieb’rkq MC—b’rk cb—“bdZlV:Í:ŽHrV`MCX\b’rka/¨©aWf”MLieMLac]LM`Í>nxÅWs>MCi[ZPmsJb—acb—^8aÍ>q MLf”MLieMLac]LMhieb—^8iCÍ

³–c—“ cr`]u›œš©oMCb— J“MLitWb’XlZurkac]LM`ÍxÎ&^8aJb“aWf”^`ieKr‰Z\b“¥`M@hieb—^8ieXCÍÏmMfgr`–J—Zhieb—^8ieXCÍЙ·¹¸c·¹¶º‰»#ÑW½eº‰·¹¸c·¹¸Y¾™Ò§º‰¶@Ó

À:»—Á¿uÍcnaœZ\ie^`sYV:ÍWhi\b“^`imOr`—b“ Jiur‰Zeb—^8aÍYnxÅWs>^8aJMLaœZ\b’rk6Í Œ MLb“ J–J“¯¦

∗ ԔÕ$ÖÔ¹×S؉ّگِÛgÜ\ÝpÞ$ßlà‰áu۔گâ[ã âläÚåâNæ#áeÚ¯çßlã&áeÚ¯èêéCِâlÜ[ÝëäèìâlÛgÜ6èÚ¯ßNíJáuÛgèêܯîgïCّåœÝð‘ñlòó¤ôõ$Û¯Ü6á[ö˜÷:âlåâ©ø8Ýâlã&áuèêùHú äèêûlü¤ùýáuÜ\þÿkü¤ÙÜ6éLِâ©Ú$ã&áeÚ6çœþÙCî¹àÜ6ّåœþ“Û

21/ ) §$N £& / „ p¤

… ‹ –cr`actŸ“Mms>rkiur`tWb“_`KNM >rVCX\b“MLa„tW^8bdZLZ\ieMmKNb’XHMLa#–c¥œieM±s>^8–JiHMCX[Z\b“K@MCiH—Mscr`ier`K@ILZ\ieM

t! –Ja®KN^YtJIL“M/tWM£t"‘]b’X[b“^`ažx’r*K@^Wt"C—b’Xer‰Z\b“^`a º„À,½¤·g‰½¤·tWM£]M/scrkiurkKNIZei\MtW^`b—Z#LZ\ieM/]Crk“b— ci$CMZ\ieICX

X\^`b“_`acML–cX\MLKNMLaœZ‘ž§scrki\Z\b’]–J“b“ILieMLKNMLaœZ#tJrkacX–ca´]Cr`tWieMNtWM@¡>rk Jb““bdZeb%b“actW–cX[Z\ieb“ML“—MN^&’r/]L^`aJa>rkb’X\Xerkac]LM

X\–J WjlM‘]¤Zeb—¥8MMCX[ZZei\I‘X–JZ\b“—b’X'CM`¦TYb$“r„¥Yier`b“X\MLK( J“r`ac]M(tWMCX±tW^`aca(LM‘X^8 cX[MCi\¥)LM‘X MZ“r]^8aJacrkb’XeX\r`ac]M„º

À:½¤·g‰½¤·cX\M&i*L¥8IL“MLaœZMLaNZei\^8s@_8ier`actNt"CXer`]C]^8ietNX\–Ji’r#ªL^8aJMtWM]L^`aW¡,rkac]LM&•8–+ MC—“MCXpb—actJ–Jb“X\MLaœZps,^`–ci]M

s>rkiurkKNIZei\M8žk, b“aWf-LieMLac]LM  crV‘X[b“MLacaJMHs>MC–WZ]L^`actW–cb—ieM/.#tWM Krk–J¥‰r`b“X\MCXt(C]b’X\b—^8acXL¦0or#t(Z\M‘]¤Zeb—^8aNt! –Ja

ZeML,]^`aJ°cbdZi\MC¥Z–caŸr`X\s>M‘]¤Zb“K@s,^`i\Zer`aœZšMZ]–Jieb—MC–cX\MLKNMLaœZX\^`–cX[š<MCX[Z\b“K#š$MLaŸX[Zer‰Zeb“X[Z\b’•œ–JM&rkscsJ—b’•œ–(LM

.(tWMCXsJie^` J“ILKNM‘X]L^`ac]Li\MLZeXC¦$ϱrkacXH]LMiurksJs,^`i\ZaJ^8–cXsJi\^8s>^œX[^8acX0 MLKNsJ“^`b§t1 –Ja]Li\b—Z\ICi\M

Coh(π;Xⁿ)

KNM‘X[–JiurkaœZ“r@]^8K@s>rkiurk Jb“—b—Z$mMLaœZ\ieM’r(K@^Wt"C—b’Xer‰Z\b“^`aºmÀ,½¤·gÂk½¤·

π ^MZH, b—aJf”^`ieKNrkZ\b“^`a/rksJs,^`i\Z$CMscr`i“MCX tJ^`aJa(CMCX

Xⁿž,scrki&–ca™ier`sJs>^8i[ZtJM(tWb’XlZurkac]LMCX&tJM³–J“— cr8]u›œš<oMLb“ J—MCiC¦HTYb

Coh(π;Xⁿ)≤1]M‘XtWMC–WÅ b“aWf”^8i\Kr‰Zeb—^8acXpX\^`aœZ t"C]L“r`i$CMCX]^`G2LieMLaœZ\M‘XL¦$h0^`–Ji, –WZeb—“b“X\MLiHtWM±KrkaJb“ILieMm]L^`ac]Li\ILZ\M`žœaJ^8–cXHtWML¥8^`acXt"š

¡>aJb—i0–JacMH]L“r8X\X\MHt1 ’ºHÀ,½¤·gÂk½¤·œsci\^8sJi\M‘XLž‘fgr`b— J“MLKNMCa8Zx^8–(aJ^8ab“aWf”^`ieKr‰ZebdfgXCžkieMLsJi*CX\MLaœZer`aœZ0 r`sJs>^8i[Zt1 –JaJM

]L^`aJa>rkb’X\Xerkac]LMNK@b“aJb“Krk“MŸ]^`G2LieMLaœZ\MŸr¥8MC]@—M‘XtW^8aJa(LM‘XL¦£Î&^8–cX#sJie^`s,^8X\^`a>Xs>^8–Ji]MNfgr`b—ieM–JaJMt"š

Kr`ie]uGJMf”^`act"CMX\–Ji$, –WZ\b““b“Xer‰Zeb—^8at1 C]uGcr`a8Zeb—““^`acX$t1 rksJsJieMLaœZeb“XeX\r`_`MxKNb“aJb—Kr`–WÅ#MZ0sJ—–>X_)La(Cier`—MCK@MCaœZ

X\–Ji#tWM‘X#—^8b“XºNÀ,½¤·g‰½¤·&º(Àc¿¤¼©Á½¤·gÂk½¤·¹¦ŸnaW¡>až–JacMŸtWb“Xe]–>X\X\b—^8a˜_)La(Cier`—MN]43kZe–Ji\MN]Miurkscs>^8i[Z‘ž§b—aœZ\ie^kš

tJ–Jb“XerkaœZ—M±]iebdZeILieM

Coh]^8KNK@Mm–JaŸ^8–WZ\b“§tWMm]Crk“b— Jiur‰Zeb—^8aŸt1 –Ja*ºÀ,½¤·gÂk½¤·>X\–J WjlM‘]¤Z\b—fl¦o5 –WZeb—“b“Xer‰Zeb—^8a„tWM ]LMm]Li\b—Z76MLieMX\MLiur#b—““–cXlZei$CMscr`i0 rksJsJ“b’]Lr‰Zeb—^8aŸrk–Wŝ“^`b’XpMLÅWs>^8aJMLaœZ\b“ML““MCXMLZHtJM Œ MLb“ J–J“¯žJ]LMLi\Zerkb“aJMCK@MCaœZ

“MCXHsc—–cX&–WZ\b“—b’X$CXHMLa/¡>r` Jb“—b—Z$#s,^`–Ji KN^Wt"C—b’X[MCi&tWMCX tJ–Ji$CMCX tWM#¥Yb“M`¦

8Ë

†‘…

É

Š9

…Ä‹

¡>r` Jb“—b—Z$™b“actW–cX[Z\ieb“ML“—MÍ&b“aWf-Ci\MCac]M™ crV)CX\b—MCaJaJMÍ&^`scb—aJb“^`a t1 MÅWs,MLi\Z}Í&KN^Wt"L“b“Xer‰š

Zeb—^8a º(À:½¤·g‰½¤·ÍtWb’X[Zerka>]MNtWM³–J—“ cr8]u›8š©oMCb— c—MCiͺ@À,½¤·g‰½¤·$aJ^8a˜b“aWf”^`ieKr‰Z\b—fgXNͺ(À:½¤·g‰½¤·$^` WjlM‘]¤ZebdfgXNÍ

‘]uGcrkaœZeb—“—^8acXt1 rkscsJi\MCaœZ\b’X\Xerk_8MKNb“aJb“KNr`–Wţ͜MCaœZ\ie^`sJb“M#ÍW]Lrk“b“ JierkZ\b“^`a}º&À,½¤·gÂk½¤·ÍYK@^WtWIC—MMÅWs>^8aJMLaœZeb—MC¯Í

KN^WtWIC—M#tWM Œ MLb“ J–J“

1¤±1

KN^`aJ_ÄZeGJM²rki\Z\b’]“MCX}^`a X[–c WjlMC]Z\b“¥`M«ŽHrV`M‘X[b’rka b—aJf”MLieMLac]LM´Z\^ M‘XlZeb—Kr‰ZeMZ\GcM«scrkiurkKNMLZ\MLi

θ ^{^`fNr}

tJMC]b’X\b—^8aWš6Kr`›œb“aJ_ŸKN^WtWMC

M(θ)ž§KrkaYV£^kfxZeGJMLK€b—acX\b’XlZ±^`a}ZeGJMNtWb“X\sJ“MCr8X[b“aJ_„ ,MLG>r¥œb“^`im^kfxZeGJMNs>^œXlš

ZeMLieb—^8i^WtJtJXm¢ GJMCa™Z\GJM(sJieb—^8ib“aWf”^`ieKr‰Z\b“^`a™b’Xm cr`tW“V™]^8actWb—Z\b“^`aJaJM‘t§¦mŽV/Z\Gcb“XZ\MCi\Kµ¢MMLa>]^`KNscr8X\X

r/¢ ie^`aJ_/b“aWf”^`ieKr‰Z\b“^`a_`b“¥`MCa* YV*MÅWs>MCi[Z#^8sJb—acb—^8acX”^8irkaYV™^kZeGJMLi(X[^8–Jie]LM(^`fHX\–J WjlM‘]¤Zeb—¥8M@b“aWf”^8i\Kr‰š

Zeb—^8a r`act™rsJieb—^8i&tWb’XlZei\b“ J–WZeb—^8a

π(θ) fgrki f”ie^`K Z\GJMf”i\M‘•œ–JMLaœZ\b’X[Z]^`aJ¡>tWMLa>]MieML_8b—^8a/^8a™scrkiurkKNMLZ\MLi

θ  Jie^`–J_8G8Z( œV´^8 cX[MCi\¥8MCttJrkZer

Xⁿ = (X1, . . . , Xn)¦FHGJM/^` JjlMC]¤Zeb—¥8M„›YaJ^‰¢ “MCtW_8MŸ^kf ZeGJM/i\M‘rk“bdZlV _8b—¥8MLa/ YVŸZ\GcM—b“›`MC—b“GJ^Y^Wt

L(Xⁿ, θ) ]Lrka/ZeGœ–>X  >M(tWi\^‰¢ a/ YVŸZeGJM]uGJ^`b’]M#^`f0rN¢ ie^`ac_@sci\b“^`i&r`actZ\GJM

ŽHrV8MCX\b“r`a(rka>rk“VYX[Zp]Lr`a@Zerk›8M&rs>^œXlZeMLieb—^8itWMC]Lb“X\b“^`a@¢ bdZeG–JaY¢MC“]L^`KNM&]^8acX[M‘•œ–JMLac]LMCXC¦¨©aKN^8X[Zp]Lr8X[M‘XLž

ZeGJMCX\Mr`i[Zeb“]L—M‘XxrkieM&]MLaœZeMLieMCt@^`aŸrieMLb“aWf”^`iu]MCKNMLaœZ^kf,Z\GJMms>^œXlZeMLieb—^8ixi\^8 J–cX[Z\aJM‘X\XC¦^‰¢MC¥`MCiCžk^8aJMH¡ciuXlZ

tJbdß]–cdZlVb“X$jl–cX[Z]^8acX\b“tWMCi\b“aJ_(Z\GcrkZ Z\GJMr¥‰r`“r`b— J“M±›œac^‰¢ —M‘tW_`Mb’XHaJ^kZX\ML—f¹š<]L^`aœZ\iur`tJb“]Z\^`ieV`¦

Î^kZ\MHZeGcr‰Zb“a]Lr8X[MH^kf,GJb“_`GJ“V]MCacX\^`ieMCt@tJr‰Zur±r`act@X[Krk“cXerkKNsJ“MCXCžk¢ GJMCaZeGJM&ŽHrV`MCX\b’rkab“aWf”^8i\Kr‰š

Zeb—¥8MrksJsci\^œr`]uG/b“X ¥8MLieV„ieMC]L^`KNKNMLactWM‘tgq ^8 >MCi[Z‘ž,~kz8z ¤žYZ\GcMGJMZeMLie^`_8MLaJMCbdZlVŸ^`f0Z\GJMtcr‰Zer]Crka£Gcr¥8M

ZeGJMŸXerkKNMNM:MC]¤ZuX#r`X#r/¢ ie^`aJ_/sJieb—^8i#¢ bdZeG´]L^`iei\M‘]¤Z#tJr‰ZurJ¦ sJieb—^8i#]Lrka ,M¥`MLieV}¢ML“]uGJ^8X\MLa* J–WZ

ZeGJMs>^œXlZeMLieb—^8ipZeGJieMCr‰ZeMLacXZe^@“MCr8tZe^@ >r`t/tWMC]Lb“X\b—^8acXC¦$¬c^`i b“acXlZurkac]LM`žYb“KNr`_`b“aJMrka/b—actJ–cXlZei\b’rk§X\VWXlZeMLK

¢ Gc^8X\MNfgrkb“—–JieMCXGcr¥`M ,MLMCa«aJ^kZeMCttW^‰¢ a´b“a´Z\GJMscr`X[ZC¦ f¹ZeMLiNX[^8KNMŸrkKNMLactJK@MCaœZeX#ZeGJM„X\VWXlZeMLK b“X

]L–Jiei\MCa8Ze—V˜¢^8i\›Yb“aJ_c¦ KrkZ\MLieb’rkxMÅWs,MLi\Z¢ b“—_8b—¥8MNsJi\^8 crk c—MNfgr`b—“–JieMNZ\b“K@M‘X#b“a´Z\GJMNf”–WZe–JieMŸ¢ GJb’]uG

]Crka/ >Mieb—_8GœZ[š©X[GJb—f¹Z\M‘tŸf”ie^`K Z\GJMtJrkZercžœ ,MC]Crk–cX\M^kfZ\GJMZeMC]uGJaJb’]Lr`§ML¥8^`“–WZ\b“^`a¦0¨©aZ\GJb’X&]Cr`X\M`žWX[^8KNM±^`f

ZeGJMf”^`ieKNMCX[ZtJr‰Zur@KNrV„ >M]L^`acX\b’tWMLieMCtr`X s,^`“—–JZ\b“aJ_NZ\GJM#^8 WjlMC]Z\b“¥`M›YaJ^‰¢ “MCtW_8M`¦pTY–c]uG™rNX\bdZe–cr‰Zeb—^8a

K(–cX[ZH ,M#aJ^`Z\b’]MCt/ ,Mf”^8i\M#rk“§s,^8X[Z\MLieb“^`i r`]C]MLsJZer‰Zeb—^8a¦

FHGJMCaž0r£sJi\b“^`i#›YaJ^‰¢ “MCtJ_`Mrkact´r£ieMCr`›YaJ^‰¢ —M‘tW_`MN]Lr`a´Gcr¥`MX\VœKNKNMZei\b’]Nie^`“MCXb“a«r™]^`aJ°cb“]ZC¦

nX[s,MC]Lb“r`—“V²b“a rkaÄb—a>tW–cX[Z\ieb“r`]^8a8ZeMÅYZCžZ\GcM™sJi\b“^`iŸ]Crk“b— cierkZ\b“^`aÄrkacX\¢MCieX(Ze^´Z\GJM˜X\Gcr`›`ML^8–WZ^kf±Z\GJM

^8 cX\MLie¥`MCt/tJrkZerg œV/ieMLKN^‰¥Yb—aJ_N^8–WZ\“b—MCieXCžc c–Ji\aJš6b“a}tJr‰ZurJž>MZe]`¦ ¦O–Ji\b“^`–>X[“V`žcZ\GJb’XX\–J WjlM‘]¤ZmX\MLMCKNXaJ^kZ

Ze^™Gcr¥`MN ,MLMCaK–c]uGX[Z\–ctWb“MCtr`dZeGJ^`–c_`GbdZ(]L^`acX[Z\b—Z\–JZ\MCXCžf”i\^8K€^`–cis>^8b—aœZ^`fH¥Yb“ML¢#ž$rka´b—KNs,^`i\Zer`a8Z

sci\MC—b“KNb—acr`i\V´Ze^«ŽHrV`MCX\b’rka²X\–J WjlM‘]¤Z\b“¥`M£b“aWf”MLieMLa>]M`¦

XNfgrkiŸr8XN¢M£›YaJ^‰¢#žpb—Z„tW^YMCXNac^kZŸMÅWb’XlZŸX\b—K@š

sc—M@XlZur‰Z\b’X[Z\b’]Z\^Y^`’X f”^8i±tJM¡caJb“aJ_ZeGJM(r`_`ieMLMCK@MCaœZ >MLZl¢MCMLa}r„sJi\b“^`imtWb“X[Z\ieb“ J–WZ\b“^`a*rkact™r`a}^` WjlM‘]¤Zeb—¥8M

›YaJ^‰¢ “MCtJ_`M± Ji\^8–J_`GœZH YV„^` >X[MCi\¥8MCt„tcr‰Zerc¦

Î^kZ\b’]MGJ^‰¢ML¥8MLiZeGcr‰Z0b“aZ\GcMX[Z\ieb“]Z$^8 cX\MLie¥rkZ\b“^`a#^`fJZ\GJMHŽHrV`M‘X[b’rka#scr`ier8tWb“_`K£žCsci\b“^`i0tWb’XlZei\b“ J–WZeb—^8a

r`act£“b—›8ML“b—GJ^Y^Wt™]^8acX[Z\b—Z\–WZeMr¢ GJ^8—MtJMC]b’X\b—^8a™K@^WtWMCr`act£bdZX[MCMLKX&rkZ¡ciuXlZ±X[b“_`GœZ&aJ^`ZmieML“ML¥‰rkaœZ Z\^

tJMZ\MCi\KNb“aJMNtWML_8i\MCMCXm^kftWb’Xe]ieMLscr`ac]V/ >MLZl¢MCMLa˜Z\GJMCK/žf”i\^8K rZ\GcML^`ieMZeb“]Crk0s,^`b“a8Z^kf¥œb“ML¢#¦FHG>r‰Zb“X

¢ GYV}Kr`aœVŽHrV`MCX\b’rka*r`i[Zeb“]L—M‘X#rkieM]MLaœZeMLieMCt*^`aXlZe–ctWVYb“aJ_/Z\GJM„r`_`ieMLMCK@MCaœZ±^`fZeGJM„tJrkZer/¢ bdZeG´Z\GJM

ŽHrV8MCX\b“r`a}KN^YtJMLx]uGJ^`b’]MNrka>t}ZeGJMtWMZeMC]¤Zeb—^8a*^`f¿W½6À,½¤·’¿LÁL¦NTWMLM@f”^`i#b“acX[Zerka>]MNŽrVœrkiei\b$rkactŽMLie_`MCi

Cy8yYžCy8y ¤¦F^^8–JiH›œac^‰¢ —M‘tW_`M8ž`ZeGJMLieMmr`i\Mmf”ML¢ r`i[Zeb“]L—M‘Xp¢ Gc^8X\MmX\–J WjlMC]Zb’XZ\GJM#tWMLZ\M‘]¤Z\b“^`a^kf]L^`aWš

°>b“]Z± ,MZl¢MLMCa˜b“aWf”^`ieKr‰Z\b“¥`M@sJieb—^8ieXmrkact*tJr‰ZurJ¦#¨©a*f”^`ieKNMLi±¢^8i\›WXCž>b“actWb’]LrkZ\^`iuX±rkieM(–cX\–cr`—“V£ J–Jb“dZZ\^

r`—“^‰¢ Ze^ŸKN^WtWbdf”VZ\GcMsJieb—^8i ^`i ZeGJM“b—›8ML“b—GJ^Y^Wt/ YVtWMLZ\MC]Z\b“aJ_„r`act/i\MCK@^‰¥Yb“aJ_N^`–WZe—b“MLiuX ^8i Z\^Y^b“aW°c–Wš

MCaœZ\b’rktJrkZerc¦

aJ^`Z\GJMCi&b“XeX\–JM#b“X Z\^ML“b’]b—Z&i\^8 J–cX[Z&sJi\b“^`iuX¢ bdZeG™aJML_8—b“_`b“ J“Ms>^œXlZeMLieb—^8i b—KNscr8]¤Z&¢ GJMCa£r

]L^`aW°>b“]Zb’XHtWMZeMC]¤ZeMCt§¦{˜rkb“a„ieMf”MLieMLa>]MCXrkieM±ÏmM¬b“aJMLZ[Z\bCy8 žœÏ±r¢ b’t‘yœ~ ¤ž &b““Cy ! o–>]Lr`X

Cy8y`" mrka>t*M‘X[s,MC]Lb“r`—“VP r`_8rka#‘y‰ycž$Cy"Jž$‘y`y8zJž~kz8z` r`act

aJ_8MLiuX¯~kz8z`z ¤¦@¨©ab“actW–>XlZei\b’rk

X[Z\–>tWb—M‘XLž$¨lt"CMMZ/ºk»&%¯~`z`z mGcr¥`MX\–J_`_8MCX[Z\MCt˜Z\^˜–cX[MŸr™¬$b’X[GcMLi#ZeMCX[Z >MLZl¢MCMLa´sJieb—^8irkactMCK@scb—ieb“]

KNM‘X[–JieMCX^kf–Jac]MCi[Zurkb“aœZlV@r`act„r`aºJÂr`sJsJie^8r8]uGGcr`X ,MLMLasJie^`s,^8X\MCt YV X\–JieMCrk– ¯~kz8z x¢ b—Z\GWš

^8–WZ&XlZur‰Zeb“X[Z\b’]Lr`Yjl–cXlZebd¡,]Lr‰Zeb—^8a¦

Œ MsJie^`s,^8X\MZ\GcMNf”^`“—^‰¢ b“aJ_]Li\b—Z\MCi\b“^`a˜f”^`i#tWMLZ\MC]Z\b“aJ_}rs>^`Z\MLaœZeb“r`]^`aJ°cb“]Z# >MLZl¢MCMLa˜sci\b“^`i#rka>t

tcr‰Zerc¦@ÏmMLac^kZ\M

π Z\GJMsJie^`s,^8X\MCt˜sJieb—^8i±tWb’X[Z\ieb— J–JZ\b“^`a´^`a

覟TY–JsJs,^8X\MN >MCb—aJ_™r` J—M@Ze^£tJM¡caJM

π^{M IA}

ZeGJMŸtWMCacX[b—ZlV}^`fr¶(·¹¸>·¹¶Nºk»}·¹¸Â‰½¤¶º‰¼6·Á `ºk¼<ºkÓ[ºL¾`½\ÁuÁ g{™¨ sJie^`s,MLisJieb—^8itJb“X[Z\ieb— c–WZ\b“^`až¢ GJb’]uG

b’tWM‘rk“—VieMLsJieMCX\MLaœZuXZeGJMmKNb“aJb“KNr`§›YaJ^‰¢ —M‘tW_`Mf”i\^8K rsci\b“^`i^`sJb“aJb“^`ažYb“a/r`_`ieMLMCK@MCaœZ¢ b—Z\GZ\GJMtJr‰ZurJ¦

FHGcMLa£]^`KNsJ–JZ\MZ\GcM#ierkZ\b“^

Coh(π;Xⁿ) = KL π^{M IA}(.|Xⁿ)||π KL(π^{M IA}(.|Xⁿ)||π^{M IA})

¢ GcMLieM

KL(π1||π2) b’XHZeGJM³–J“— cr8]u›œš<oMLb“ J—MCitJb“X[Zer`ac]M ,MZl¢MLMLa™tWb’X[Z\ieb— J–JZ\b“^`acX

π1

rkact

π2

KL(π1||π2) = Z

Θ

π1(θ) logπ1(θ) π2(θ) dθ.

Coh(π;Xⁿ) b’Xi\MC_8r`ietWM‘t}r`X#r£tJrkZerkš<r`_`ieMLMLKNMCa8Z±]iebdZeMLieb—^8a¦(¨<f

Coh(π;Xⁿ)≤1žsJieb—^8i#rkacttJr‰Zur‰š

_8b—¥8MLaN]^8aW¡>tWMCac]MHieML_`b“^`a>X$f”^`i

θrkieM ]“^8X\MHMLaJ^8–J_`GNrkact(ZeGJM sJieb—^8ixsJi\^8s>^œX\r`

π b’Xxb“ark_`ieMLMCKNMLaœZ$¢ b—Z\G ZeGJM#^` JjlMC]¤Zeb—¥8M±tcr‰ZerN›YaJ^‰¢ “MCtJ_`M`¦$n’X[MrN]L^`aW°cb’]¤Z&b’X tWMLZ\MC]Z\M‘t§¦

¨©a TYM‘]¤Z\b“^`aÄ~*^kfZeGJb’Xrki\Z\b’]“M`žp¢M™tWMZurkb“HZ\GJM™r`i\_8–JKNMLaœZeX@¢ GJb’]uG®“MCr8tZe^*ZeGJM}]uGJ^8b“]LM^`fZ\Gcb“X

]Li\b—Z\MCi\b“^`a¦&PmacM^kfxZeGJMKrkb“a}tWb—ß]–J—ZlV£ier`b“X\MCt/ YVb—ZeX±tWML¡caJb—Z\b“^`a}b’X&_8b—¥Yb“aJ_„rsJieMC]Lb“X\MX\MLacX\M#Z\^„Z\GJM

{}¨

tJb“X[Z\ieb— c–WZ\b“^`acXC¦ TYMC]Z\b“^`aāb“X@ZeGœ–>XNMLaœZ\b“i\MC—V®tWML¥8^kZ\M‘t«Ze^]^`a>X[b’tWMLiur‰Zeb—^8acX(^`a®Z\GJM}]uGJ^8b“]LM^`f

{}¨

]Lr`actWb’tJr‰ZeMCX(rkactM‘XlZeb—KrkZ\b“^`a«^`f

Coh(π;Xⁿ)¦TY^8KNMŸi\MC—MC¥‰rkaœZ@tWb—ß]–J—Z\b“MCX@rkieMŸMC¥`^`›8MCt´rka>t

“b“aJ›WX ¢ bdZeG£MÅWb“X[Z\b“aJ_aJ^8aJb“aWf”^`ieKr‰Z\b“¥`MsJieb“^`iuX rkieMtJMCXe]ieb— ,MCt§¦FHGJM#Krkb“a/s,^`b“aœZ^kf$Z\Gcb“X&X\MC]Z\b“^`a£¢ b“—

 ,M#ZeGJM–cX\M^`fH¶@·¹¸c·¹¶º‰»0¼6½\º‰·¹¸>·¹¸œ¾„¿Lº‰¶À,»—Á¿¤ž§tWM¡cacMCt™b“a£Z\GJM@]^8a8ZeMÅYZ^`fxŽHrV8MCX\b“r`aKN^WtWML$X[MC—M‘]¤Zeb—^8a

 YVŽMLie_`MCi(rka>th0MLiei\b’]uGJb Cy8y`cž0~`z`z8~" ¤žxrka>t«KN^`ieMŸ_`MCaJMLiurk““V*^`f&Z\GJM/aJ^`Z\b“^`a«^`fHÀJ‰¿u¼©ÁL½¤·g‰½(À,½¤·gÂk½

tJML¥`MC—^8s>M‘t œV*h LieMLª‘y`y"" ±rkact´h Ci\MCªŸrkact´ŽMLie_`MCi ¯~kz8z8~" ¤¦ f¹Z\MLie¢HrkiutJXTYM‘]¤Z\b“^`a!™b’XtWM‘rk“b—ac_

¢ b—Z\G„Z\GJMrkscsJ—b’]LrkZ\b“^`aŸ^`fsJieML¥Yb“^`–cX\—VNtWML¡caJMCt„i\–c—M‘Xp^8aŸZ\GcMmMLÅWr`KNsJ—M‘Xp^`fMÅWs,^`aJMCaœZ\b’rk:rkact Œ MLb“ J–J“

KN^WtWMC“XC¦±FHGcM(]^8ac]“–cX\b—^8a}ML¥8^`›8MCXHZeGJM(–>X[M(^kfZeGJM

Coh XlZur‰Z\b’X[Z\b’]#Z\^„Z\GJMN]Lr`—b“ Jiur‰Zeb—^8a™^kfpX\–J WjlMC]Z\b“¥`M sci\b“^`i tWb’XlZei\b“ J–WZeb—^8acXC¦Pms>MCab’X\X\–JM‘X rkieM±sci\^8s>^œX[M‘tŸb“aZeGJb’X ¢rV8¦

!"$ &%' ()*,+ .-/"$L1#"­2.2/

¨©aZeGJb“X(X[M‘]¤Zeb—^8a¢MŸrkieMNb—aœZeMLieMCX[Z\MCtb“a´¡cactWb“aJ_}r™]Li\b—Z\MCi\b“^`ark““^‰¢ b—aJ_/–cXZ\^™sJieMCtWb’]¤Zr™X\VYKNK@MLZ\ieb“]

b“ac]L^`GJMCi\MCac]M f”^8iZeGJMm]L^`KNscrkieb’X[^8a@^`fr#sJieb—^8itWb’X[Z\ieb— J–JZ\b“^`a

π(θ)r`acttJr‰Zur#f”^8ir#scr`ier`KNMZ\ieb’] KN^YtJML

M(θ)¦&FHGJM@]uGJ^`b’]M#^`frŸX[–c WjlMC]Z\b“¥`MsJieb“^`itWb’X[Z\ieb— J–JZ\b“^`a

π(θ)]Lr`a™ >Mjl–ctW_`M‘t£ YV„ZeGJM(]L^`KNscrkieb’X[^8a  ,MZl¢MLMCa

π(θ) r`act^kZ\GcMLisJieb“^`iuX±]L^`acX\b’tWMLieMCtr`X#b“ark_8i\MCMLKNMLaœZ#¢ bdZeGZ\GJMŸ^8 cX\MLie¥`MCt*tJr‰Zurgr`_`ieMLMš

KNMCa8Z[ ,MLac]uGJKr`i\›WX ž§f”^`iZeGJMŸKN^WtWML

M(θ)¦„FHGY–cX¢MC—x›YaJ^‰¢ aZ\^Y^8“X^`fH]L^`KNscrkieb’X[^8a˜ ,MZl¢MLMLa tJb“X[Z\ieb— c–WZ\b“^`acX ]Crka/ >M#–>X[M‘t§¦FHGJb’X b“XZeGJMX[Zerki\Z\b“aJ_Ns,^`b“a8Z ^`f$^8–Ji Krkb“ab’tWMCrc¦

oMZ

Xⁿ = (X1, . . . , Xn)  ,Mb“actWMLs,MLa>tWMLaœZ\“VSrkact b’tWMLaœZ\b’]Lr`—“V tWb’X[Z\ieb— J–JZ\MCt ieMCr`dš<^`i\š6¥8MC]Z\^`i\š

¥‰r`—–JM‘t´ier`actW^8K ¥‰rkieb’rk J“MCX#b“aZeGJMXerkKNsJ“M„X\scr`]LM

S ¢ bdZeG²r™sJie^` >rk Jb“—b—ZlV´tWMLacX\b—ZlV˜f”–Ja>]¤Z\b“^`a ”s:tYf

p(x|θ)¢ b—Z\G£ieMCX\s>M‘]¤ZHZ\^ŸrNtW^8KNb—acrkZ\b“aJ_NKNMCr`X\–JieM

µ^žcrkact θ∈Θ⊂IR^{p r} p−tWb“KNMLacX\b“^`acr`§scrkiurkKNMš

ZeMLi&¥8MC]Z\^`i‘¦xoMLZ

Π  >MZeGJMX\MZ&^`f0r`—§sJieb“^`i KNMCr8X[–JieMCXH^8a

θ<sup>r`act</sup> Π<sup>I</sup> ⊂Π ZeGJMX[–c cX[MLZ&^kfxrk“sJie^`s,MLi g^`iŸ·¹¸Â‰½¤¶º‰¼6·ÁsJieb—^8ieX^`a

Θ ^”b6¦M8¦ Π^I = {π ∈ Π, R

Θπ(dθ) = 1} ¤¦£Î&^`Z\MZeGcr‰Z(¢MŸ]L^`aWf”^8–Jact X\^`KNMLZ\b“K@M‘X±Z\GJMac^kZerkZ\b“^`a

π ^kfr£sci\b“^`i#tWMCacX[b—ZlV˜¢ b—Z\GZeGJMŸ]Lr`aJ^`aJb’]@aJ^kZur‰Zeb—^8a*^`fZeGJMŸtWb’XlZei\b“ J–WZeb—^8a

¢ GcMLa˜ac^£]L^`aWf”–>X[b“^`a]Lrka*i\M‘X[–cdZ ¤¦FHGJMNaJ^`Z\b“^`a*^kfpb“aWf”^`ieKr‰Zeb—^8a}f”^`i#rsJieb—^8itJMLacX\bdZlV™b’X±i\MC“rkZ\M‘t™Z\^

r„KNMCr`X\–JieM^kf–cac]MCi[Zurkb“a8ZlV8¦ sJieb—^8i

π1

b’Xb“aœZ\–Jb—Z\b“¥`MC—V£KN^`ieMb“aWf”^8i\Kr‰Zeb—¥8M#Z\G>rka*r„sJieb—^8i

π2

b—fZ\GJM

–cac]MCi[Zurkb“a8ZlV@^kf

π1

b’XX[Krk““MLiZeGcrka„Z\GJM

π2

X^`acM`¦pPmaJM^`f§Z\GJMmKN^8X[Z–cX\MCt„K@M‘r`X\–JieM&^kf–Ja>]MLi\Zer`b—aœZlV

f”^8i&rNtWMLacX\b—ZlV

π b’XHZeGJMtWb&,MCi\MCa8Zeb“r`§MLaœZ\ie^`sYV

−R

Θπ(θ) logπ(θ)dθ^¦

TY–csJs>^œX[M@Gcr¥Yb“aJ_„rŸie–J“MZe^tWML¡caJM

π^{M IA}∈Π^I r8XKN^WtWMC—“b—ac_ŸZ\GcM(“MCXeXb“aWf”^8i\Kr‰Zeb—¥8Mr¥‰rkb““r` J“M sci\b“^`i(^`sJb“aJb“^`a^`a

θ^žxb—a rk_8i\MCMLKNMLaœZ¢ b—Z\G«ZeGJMtcr‰Zerc¦

tWb“Xe]–>X\X\b—^8a²rk ,^`–JZ@X\–c]uG«r}ie–J“Mrka>t´Z\GJM

X\ML“MC]Z\b“^`a}^`f

π^{M IA} b“XtW^8aJM@b—a*Z\GJM@aJMLŜZ#X\MC]¤Zeb—^8a¦

XeX\–JKNb—aJ_

Θ Ze^ ,M@r/]^8KNscr`]Z±X\–J cX\MZ^kf

IR^p

¢ Gc^8X\M( ,^`–Ja>tJXtWMCs>MCact^`a˜MLÅYZ\ieMLKNMN¥r`—–cMCX^kf

Xⁿž$r/_`^Y^Wt}MÅJr`K@sc—MNb’X#r–caJbdf”^8i\K€sJi\b“^`i^8a

Θ^ž

¢ Gcb“]uG/Kr‰ÅWb—KNb“ªLM‘XZ\GJM#MCa8Zei\^8sYV`¦

¨©af”ML¢ ¢^`iutJXCžœZ\GJb’Xi\–J“M]Lrka >Mb“aœZ\ie^YtJ–c]M‘t„r`Xpf”^`““^‰¢ b—aJ_

π^{M IA} ieM°>MC]¤ZuXX\^(f”ML¢ b“aWf”^8i\Kr‰Zeb—^8a ZeGcr‰ZHZeGJM#s>^œXlZeMLieb—^8i tWMLacX\b—ZlV

π^{M IA}(.|Xⁿ)b“XH¥8MLieVŸ]“^8X\M±Z\^@Z\GJM#s,^8X[Z\MCi\b“^`i tWMCacX\bdZlV

π^J(.|Xⁿ)^{¢ GJMCa} π^J b’X&rNaJ^8aJb—aJf”^`ieKNrkZ\b“¥`MsJieb—^8iC¦xFHGJb’X s,^8X[Z\–J’r‰ZeMX[MCMLKXHZe^@ ,Mr@ieMCr`X\^`a>rk J“M±r`act/b—aœZe–JbdZeb—¥8M#^`–WZuX[MLZCž

r8X\X\–JKNb“aJ_

π^J _8MZuXHs>^œXlZeMLieb—^8iH]L^‰¥`MCier`_`M&sJie^`s,MLi\Z\b“MCXH¢ b—Z\G™r`a/r8]L]–cierkZ\M±f”i\M‘•œ–JMLaœZ\b’X[ZH¥‰r`—b’tWb—ZlV`¦

FHGJMCažJrX\–J WjlM‘]¤Zeb—¥8MsJieb—^8i^8sJb“aJb—^8aŸb’Xs,MLi\f”MC]Z\“Vb—a/r`]C]^`iutJr`ac]M ¢ b—Z\G„Z\GJM±^` WjlM‘]¤Zeb—¥8M›YaJ^‰¢ “MCtW_8M

_8b—¥8MLa œV„Z\GJMtcr‰Zer@^8a

θ b—fZeGJMf”^`“—^‰¢ b“aJ_@GYVœs,^kZeGJMCX\b’X b“XH¥8MLiebd¡>MCt§¦

@Ñ cÁ À:½¤·g‰½¶Â`ÁL»¹»ý·¹¸œ¾·’¿ÀJÁL½Á ¼6»Ÿ·¹¸´ºL¾`½\ÁuÁ¶ÁL¸c¼x·¹¼}¼cÁ(¤¿LÁL½ Á `º‰¼©º

X^{n ·} π(θ) = π^{M IA}(θ|Xⁿ)∝ L(Xⁿ, θ)π^{M IA}(θ).

¨©a£X\–J WjlMC]Z\b“¥`M#ŽHrV`M‘X[b’rkaŸb“aWf”MCi\MCac]M8žW^`aJM^kfZ\GJM#Kr`b—a/b“XeX\–JMCXHb’XZ\^]^8aœZ\ie^`,Z\GJM#b“aWf”^8i\Kr‰Zeb—^8aŸZeier`acXlš

’r‰ZeMCtÄb“a Z\GcM˜sJieb—^8i

π^¦ ÏmM¡>aJb—ac_

π^{M IA}(.|Xⁿ) r8XZ\GcMuÁ¤¿¤¼@·¹¸ Âk½¤¶Nºk¼¯·‰Á/sJieb—^8i„]uGJ^`b’]M8ž ¢M*rkieM b“aœZ\MCi\M‘XlZeMCtb—a£KNMCr8X[–Jieb“aJ_ZeGJM·¹¸ Âk½¤¶Nºk¼¯·‰Á(½\Á<¾k½eÁ¼

KL π^{M IA}(.|Xⁿ)|| π

r`act´X[MC—M‘]¤Zeb—aJ_™tWMCacX\bdZeb—M‘X

π sJi\^WtW–>]b“aJ_£¢M‘rk›}¥‰rk“–JM‘X#^kfZ\GJb’X#ieML_8i\MLZC¦¨©actWMCMCt§ž

KL(π1||π2) X[Zer‰ZeMCX ZeGJMŸieML_8i\MLZtW–JMZe^™Z\GJM]uGJ^`b’]M^`f

π2

¢ GJMCa´Z\GJMŸZ\ie–JM„tWb’XlZei\b“ J–WZeb—^8ab“X

π1

¦ Œ MC—tWM¡cacMCtaJ^`Z\b“^`acX

^`fxi\MC_`ieMZ&¢ b—Z\G}b“aWf”^`ieKr‰Zeb—^8aWš¯ZeGJML^8i\MLZ\b’]±Z\^Y^`’Xr8X ZeGJM(³–J““ cr`]u›œš©oMLb“ J“MLi&tWb’X[Zerka>]M]Crka™ ,Mf”^`–cact™b“a

O^‰¥8MLi FHGc^`Kr`X ‘y`y rkact£TY¢MLMLZ\b“aJ_cžJϱr‰Z\Zer Q±Gc^8X\G 6~kz`zœw ¦

sJieMLKNb’X[M±^kf$Z\Gcb“XHb’tWMCr(Ze^NtJM¡caJM

π^{M IA}(.|Xⁿ)r`XZeGJM >M‘XlZ&s,^8XeX\b— J“M±sJi\b“^`iHGcr8XH >MCMLa/aJ^kZurk J“V b“aœZ\ie^WtW–c]M‘t² YV ŽMCi\a>rkiutW^ Cy`y ¦ÄFHGcMLaÄbdZŸb’XaJM‘]M‘X\XerkieV´Z\^]uGJ^Y^8X\M™r´]^`a>XlZurkaœZ

C X\–c]uGÄr`XNb—f

KL π^{M IA}(.|Xⁿ)||π

> C ^ZeGJMLa 𠝓^8X\MCXmbdZuX±tcr‰Zerkš<r`_`ieMLMCK@MCaœZC¦FHGJb“X±–Jscs>MCi± ,^`–cact}b’X±i\M‘r`]uGJM‘t

¢ GcMLa

π=π^{M IA}¦¬$b“acrk““V`žWZeGJMtJr‰Zur‰š©rk_8i\MCMLKNMLaœZ]Li\b—Z\MCi\b“^`a/rkact/ie–J“Mr`i\M_8b—¥8MLa„b“a™ÏML¡caJb—Z\b“^`a`¦

`¦(¸: x·¹¸œ¾

π<sup>M IA `Á ¸,Á¼JÁ</sup> `º‰¼©º‰Ó[ºC¾k½eÁeÁL¶Á¸>¼ L½¤·¹¼<ÁL½¤·g‰¸

Coh(π;Xⁿ) = KL π^{M IA}(.|Xⁿ) ||π KL(π^{M IA}(.|Xⁿ)|| π^{M IA})

¯~

ºk¸ º uÁ<À:¼

π º‰¿·¹¸´ºL¾`½\ÁuÁ¶ÁL¸c¼ x·¹¼™¼cÁ»ý·8ÁL»ý·J‘ £Â

X^{n ·} Coh(π;Xⁿ)≤1^%

O“MCr`i\“V`ž`ÏmM¡>aJbdZeb—^8a b’XxaJ^`Zxi\M‘XlZei\b’]¤ZeMCtZ\^±Z\GJM&³–c—“ cr`]u›œš©oMCb— J“MLixtWb’X[Zerka>]M rkact@rkac^kZ\GcMLi]uGJ^8b“]LM

b“aNZ\GcM “bdšlTYb“—¥8MLV@]’r`XeXx^kf§b“aWf”^`ieKr‰Z\b“^`aJš¯ZeGJML^8i\MLZ\b’]KNM‘r`X\–Ji\M‘X —b,rkactŸTWb—“¥`MCV`ž ‘y`8" $]Lr`a >M&Kr8tWM`¦

8–cX[Z\b—¡>]LrkZ\b“^`acXf”^8i –cX[b“aJ_b—Z&sJieMf”MLieMLaœZeb“r`—“VŸ]Lr`a ,Mf”^`–Ja>tb“a r`i[Zeb—_œrka Cy8y" rka>t£TWb—acr`aJ^‰¥Yb±rka>t

8^`GcacX[^8a¯~`z`z` ¤¦0Î&^`Z\M&ZeGcr‰ZHÏML¡caJb—Z\b“^`a m]Lr`aŸ ,MsJ–WZHb“aŸi\MC“rkZ\b“^`aŸ¢ b—Z\GŸZ\GJM@·¹¸ Âk½¤¶Nºk¼¯·gÂk¸}¼¯½eº‰¸J¿ÁL½

½eº‰¼6·g‰žtWML¡caJMCt˜ YV/’r`X[Z#rk–WZeGJ^`iuXL¦

]L]L^`iutWb“aJ_Z\^/b“a8Ze–Jb—Z\b“^`až,Z\GcM@tJrkZerkš<r`_`ieMLMLKNMCa8Z±]iebdZeMLieb—^8a}X[GJ^8–J’t

G>r¥`M±Z\^„r8]L]LMLsWZr`aœVaJ^`a>tWML_8MLaJMCierkZ\MsJieb—^8i&¢ GJMLa

n→ ∞ž> >M‘]Lrk–>X[M^`fZeGJMsJieMLMCKNb—aJMCac]M^kfxtJr‰ZurJ¦

FHGcb“X ieMCX\–J—Z]Lr`a/ ,MsJie^‰¥`M‘t„–>X[b“aJ_NZ\GcMr`X\VYK@sJZ\^kZeb“]±aJ^8i\Kr`—b—ZlV„^`fZeGJMs,^8X[Z\MLieb“^`i&tWMCacX[b—ZlV

π(.|Xⁿ)^ž

–cactWMLi]’r`XeX[b’]Lr`J]^`a>tWbdZeb—^8acXx^`f,ieML_8–J“r`i\b—ZlVf”^`i

π^ž π<sup>M IAr`act</sup> p(x|θ)¦{}r`b—aNieMf”MLieMLa>]MCXxr`i\M8^`GJa>X[^8a Cy ‰z" ž$O’rkie›`M ŽHrkieie^`a Cy8y`z" ž$O’rkie›`M Cy8y`y r`actrki\Z\b“_8r`a! X >^Y^`› r`i[Zeb—_œrkaž Cy"`" ¦N¨©a

sJs,MLactWb—ŧžWZ\Gcb“X  ,MLG>r¥œb“^`iHb’X  Jieb—ML°cV„X[Z\–ctJb—M‘tb“a™rka/MÅWs,^`aJMCa8Zeb“r`]Lr8X[M8¦

uL! -/"$1"­2/§"$¤£„1/

¨©a Z\GJb’X™X\MC]¤Zeb—^8a ¢M˜Zei\VÄZe^ tWML¡caJMsJieb—^8isJie^`s,MLi£tWb’XlZei\b“ J–WZeb—^8acX/¢ GJb’]uG ieMLsJieMCX\MLaœZ„ZeGJM´KNb—aJb“Krk

›YaJ^‰¢ “MCtJ_`M@^kfr£X\–J WjlM‘]¤Z\b“¥`MN^8sJb—acb—^8a*b“ar8]L]L^`iutJrkac]LM(¢ b—Z\G*Z\GcMŸtJr‰ZurJ¦FHGcr‰Z#b’X±Z\^}XerV

π^{M IA b“X#r}

tJMLacX\bdZlV¢ GJb’]uG/b’X ¥‰rk_8–JMMLaJ^8–J_`G/Ze^K@^WtWMC§Z\GcM#K@^œXlZ f”–cªLªLV£]^8aW¡>tWMCac]M^8a

θž>r¥‰rkb““r` J“Mmf”ie^`K Z\GJM tcr‰Zerc¦$¬$b“ieX[Z\“V`žY¢M¢ b“—:MLÅWr`KNb—aJMZ\GJM±ML“b’]b—Zer‰Zeb—^8aŸ^kfX\^`KNM±{™¨ ]CrkactJb“tJrkZ\M‘XL¦xFHGJMCb—iHX[MLZ¢ b“—§r`—¢HrVWX

 ,MaJ^`Z\M‘t

Π^{M IA}¦HFHGcMLa™¢M#¢ b—“sci\^8s>^œX[Mie–J—M‘X&^`fxX\ML“MC]Z\b“^`a¦¨©a™scr`ier`—“ML6žcrka}r`i\b—Z\GcK@MLZ\b’]#¥`MCieX\b“^`a

^`fZeGJMŸtJrkZer‰š©rk_8i\MCMLKNMLaœZXlZur‰Z\b’X[Z\b’]Nb“X#tWML¡caJM‘t§žf”^8i#_`MCaJMLiurkx–cX\MN¢ GJMLa´aJ^}{}¨

]Lrka>tWb“tcr‰Z\M]Crka´ >M

M‘r`X\b—“VŸX\ML“MC]¤ZeMCt§¦

! #"%$'&)(+*(+$-,/.0

{}¨

]Lrka>tWb“tcr‰Z\M‘XGcr¥`M±Z\^N ,M]uGJ^8X\MLa„f”^`iHZeGJMLb“i&s,^8X[Z\MLieb“^`i ]“^8X\MLaJM‘X\X¢ b—Z\G™r@aJ^`acb—aWf”^8i\KrkZ\b“¥`M±sJi\b“^`i

π<sup>J¦³#r`XeX Œ</sup> r8X\X\MLieKrka Cy8y`" Gcr¥`M±_`b“¥`MCar(aJ^kZeML¢^`i\Z\GYVN^‰¥`MCi\¥Yb“ML¢ ^kfZ\GJM–Ja>tWMLiuXlZurkactJb—aJ_Nrka>t MC—b’]b—ZerkZ\b“^`a@^kfcac^`aJb“aWf”^`ieKr‰Zeb—¥8MsJieb“^`iuX YVX[Z\ie–c]¤Ze–Jieb—aJ_±ie–J—M‘XLž`¢ GJb“]uG@sJieMCX\MLaœZeX0X\^`KNM^kfcZeGJM ]^8ac]MCsWZeX

–>X[M‘t„b“a/Z\GJb’X&X\MC]¤Zeb—^8a¦

¨ltWM‘rk“—V8ž

π^{M IA}=π^J rkact/ZeGJM(]uGJ^8b“]LM^kf

π^J r8X ZeGJM18M:i\MCVWX sJieb—^8iMLa>X[–JieMCX&b—aY¥‰rkieb“r`ac]M#^`fp]iebdš

ZeMLieb—^8a ¯~"  YV£scrkiurkKNMZei\b“ªCrkZ\b“^`a™]uGcr`aJ_`M8¦ &^‰¢ML¥8MLi‘žcMÅJ]LMLsWZ±b—a˜tJb“Xe]ieMZeM]Lr8X[M‘XLž>Z\GJM2`M ,ieMLVWXsJi\b“^`i

b’Xmb—KNsJie^`s,MLirkact*]Lr`aJaJ^kZ± >M@–cX\MCt*X[b“ac]M(Z\GcM@tWMCaJ^`KNb“acr‰Ze^`im^kfZeGJM@]Li\b—Z\MCi\b“^`a˜b“X±tWM¡>aJMCt*–Js}Ze^r`a

–caJ›YaJ^‰¢ a/r8tJtWb—Z\b“¥`M#]^8acX[ZerkaœZ‘¦

FHGY–cX(X[^8K@MŸtJMfgrk–cdZ

π^J G>r¥`MNZ\^™ ,M„X\ML“MC]Z\MCtf”^8i#Z\GcMŸs>^œXlZeMLieb—^8i]^8K@s>rkieb“X\^`a¦£Pm–JiX[Z\iur‰Z\MC_`V b“a § <sup>c¦</sup>~Ÿ¢ b—“x >M(Z\^™]L^`acX\b“tJMLimZ\GJMCb—imÀc¿¤¼<ÁL½¤·g‰½ u ‰ÁL½\ºL¾œÁŸ¶º‰¼ Y·¹¸œ¾@sci\^8s>MCi[Zeb—M‘XL¦TY–>]uG˜sJieb—^8ieXmGcr¥8M  ,MLMCa™MCX\s>M‘]b’rk“—V/XlZe–ctWb“MCt£ YV/h$MCMLiuX ‘y`œw ¤žcQ±Gc^8X\G {™–c›`MLi¯jlMLM Cy8y`" žcϱr‰Z[Zur Q±GJ^8X\GCy8y8w"

r`act´ÏmrkZ[ZurCy8y` ¤¦ŸFHGJMCV˜sJie^‰¥Yb“tWMN^8ietWMCieX±^`ff”ieMC•œ–JMCa8Zeb“X[Z¥‰rk“b“tJbdZlV*b—a´ZeGcr‰ZX\MLacX\MNZ\G>r‰Z#Z\GcMŸs>^œXlš

ZeMLieb—^8i(]L^`aW¡>tJMLac]LM„rkieMCr™KrkZe]uGJM‘X¢ b—Z\G«ZeGJMŸf”ieMC•œ–JMCaœZ\b’XlZN]^‰¥`MCier`_`MN^`a

Θ¦˜¨©a²r™^8aJMš©tWb“K@MCacX\b—^8acrk

]Cr`X\M

p= 1 ^¤ž π^J b’X+8M:i\MCVWXL¦ Œ GJMCa

p= 2žcZ\GJM(i\MLf”MLieMLac]LM#sJieb—^8ieX&r`i\MZeGJM ,MCX[ZmKr‰Zu]uGJb“aJ_sJieb—^8ieX

¯ŽMLie_`MLi r`actŽMCi\acr`ietJ^cž Cy8y8~ ¦$Î^kZeM

Π^J Z\GJMX\MZ ^`f0X\ML“MC]Z\MCtaJ^8aJb—aJf”^`ieKNrkZ\b“¥`MsJieb—^8ieXC¦

mË …C† Ç ‡Ë ‡$‡Ë ‡‰… a>r‰Z\–cier`¯žX\b—KNsJ“M/¢rV´^kf±]ieMCr‰Zeb—ac_*{™¨ ]Lr`actWb’tJr‰ZeMCX(b“X@¢ML“H›œac^‰¢ a²b“a

ŽHrV8MCX\b“r`a/X[ZerkZ\b’XlZeb“]CX&–JactJMLimZ\GJMZ\MLieK­^kf0Àc¿¤¼<ÁL½¤·g‰½&À,½¤·gÂk½&ML“b“]LbdZur‰Zeb—^8až,r`act™]^8i\ieMCX\s,^`actJX Z\^„r`a˜r`tYš

jl–>XlZeK@MCaœZ&^kf0b—KNsJie^`s,MLiHsJieb—^8ieXHb“aœZ\^NsJie^`s,MLiHsJieb—^8ieXC¦

FHGJb’XxKNMZeGJ^WtW^`“^`_8Vb’X0—b“aJ›`M‘t(Z\^±Z\GJM ac^kZ\b“^`aN^`f¶@·¹¸c·¹¶º‰»,¼¯½\ºk·¹¸c·¹¸Y¾#¿Lº‰¶À,»“Á¤¿ g{}F&T ¤žkb“aœZ\ie^YtJ–c]M‘t

r`act/MÅYZ\MCactWM‘t£b—a£ZeGJM]^8aœZ\MÅYZ^`f0ŽHrV8MCX\b“r`aKN^WtWML$X[MC—M‘]¤Zeb—^8a/ YV£ŽMLie_`MLi r`act£h0MLiei\b’]uGJbb“a}rX[MCi\b“MCX

^`fr`i[Zeb“]L—M‘X±X\b“ac]M Cy8y`c¦(FHGJMNKrkb“a*b’tWM‘rb’X±Z\^£–cX\MNscrki\ZeX^`fpZeGJMtJr‰Zur/r8XmZ\iurkb“aJb—ac_£XerkKNsJ“MCXC¦NoMZ

π^J(θ)  >MNrka*b—KNsJie^`s,MLimsJieb—^8iC¦oMLZ

X(l)tWMLaJ^`Z\MNrŸscr`i[Z±^kfZeGJMNtJr‰Zur

Xⁿ X\–c]uG˜r`X

π^J(θ|X(l))^b’X

r„sJie^`s,MLi±s,tYfl¦Î^kZeb—aJ_

m^J(X(l))Z\GJM@Krkie_`b“acrk$tWb’XlZei\b“ J–WZeb—^8a R

p(X(l)|θ)π^J(θ)dθžrŸZeier`b—aJb“aJ_

XerkKNsJ“MNb“X#]Crk“—M‘t™À:½\ÂeÀcÁL½mb—f

0< m^J(x(l))<∞ rkact®¶(·¹¸>·¹¶Nºk»:bdfHbdZb’XsJie^`s,MLir`act˜ac^£X\–J cX\MZ#b’X sci\^8s>MCiC¦ {}F&T„b“XZeGœ–>XHZ\GJMKNb“aJb—Kr`•œ–crkaœZ\b—ZlV^kf0tJr‰Zur(f”^`i ¢ GJb’]uG/r`—§s>rkiurkKNMZeMLiuXpb“a„ZeGJM#KN^WtWML

r`i\M±b’tWMLaœZ\b—¡>r` J—M8¦

O^8actWb—Z\b“^`aJb“aJ_

π^J Z\^ŸrN{}F&T/_8b—¥8M#rk’X[^r@sJie^`s,MLi s,^8X[Z\MLieb“^`i‘žJ]^`a>X[b’tWMLieMCt/r`XrÀc¿¤¼©Á½¤·gÂk½ À:½¤·g‰½¤¦

mX[b“aJ_ŸKNM‘rkacX ^8iKNMCtJb“r`acX&^8a£{˜F&T™]^8K Jb“acr‰Zeb—^8acXCžJb—a}Z\GJM]L^`aœZ\MLÅYZ^kfxKN^WtWML$X[MC—M‘]¤Z\b“^`až>“MCr8tJXHZ\^

ZeGJMtJM¡caJb—Z\b“^`a²^kf·¹¸c¼6½¤·¹¸J¿¤·À:½¤·g‰½u¿¤¦´TW–c]uG«sJie^`s,MLi(sJi\b“^`iuXG>r¥`MZeGJMXerkKNM„r8X[VYKNsWZ\^`Z\b’]Ÿb“aW°c–JMCac]M

^8a«ŽHrV`MCX±fgr8]¤Ze^`iuXZ\Gcr`ab—KNsJie^`s,MLisJi\b“^`iuX

π^J r`act´r¥`^8b“t*Z\GcMŸsJieMCX\MLac]LMŸ^kf&r£–JaJ›YaJ^‰¢ a«]L^`acX[Zer`aœZ b“a}]Lr`“]L–J’r‰Z\b“^`a>XL¦HFHGJMLV£]Lr`a}rk’X[^ ,M(X\MLMCa™r`X&ieMCr8X[^8acrk c—M#]^8aY¥`MLaœZeb—^8acrk§sJieb“^`iuXL¦h LieMLª ‘y`y"" Hrka>t

h Ci\MCªrka>t²ŽMLie_`MCi ¯~`z`zœ~ MLŜZeMLactJMCtZeGJb’X@r`sJsJie^8r8]uGb—a²Z\GJM£X\r`KNM]^8aœZ\MÅYZN^kf&KN^WtWML X\ML“MC]Z\b“^`a¦

FHGcMLV/b—aœZ\ie^WtW–c]LM(r_8MLaJMCier`KNMCr8X[–JieM g^kf¹Z\MCa˜X\MLMLa}r8XmrŸKNr`i\_8b—a>rktWb’X[Z\ieb— J–JZ\b“^`a ^`a}r{}F&T™X\scr`]LM

χf”^8i b—aœZeML_`iur‰Zeb—ac_

π^J(θ|X(l))¢ b—Z\G™ieMCX\s>M‘]¤ZHZ\^

X(l)¦TY–c]uG£s,^8X[Z\MLieb“^`i sJieb—^8ieXHr`i\M#]Lr`—“MCtÁ ÀJÁ L¼<Á Àc¿¤¼©Á½¤·gÂk½±À,½¤·gÂk½u¿u¦¬J^`i#KN^8i\MNsJieMC]Lb“X\b—^8a*r` >^8–WZ#Z\GJMŸ{˜F&T˜r`act*Z\GJMNMÅWs,MC]Z\MCts,^8X[Z\MCi\b“^`isJieb—^8irksWš

sci\^œr`]uGžWX\MLM

actJi\b“ML–£MZ(ºk»&% ¯~kz8z pr`actŽMCi\_8MLi rkacth0MLiei\b’]uGJb6~kz`zœ~ ¦

¨©a™^8–Ji]^8aœZ\MÅYZ&^`f$sJieb“^`i KN^WtWML“—b“aJ_>žc¢M#i\MLZerkb“a/Z\GJMKNr`b—a£b’tWMCrN^`f$–>X[b“aJ_KNb—acb—Krk$•œ–crkaœZ\b—ZlV„^`f

tcr‰Zer f”^`iMC—b’]b—Z\b“aJ_±X[MC¥`MLiurkk{}¨ ]Lr`actWb’tJr‰ZeMCXf”i\^8K

{π^J₁, . . . , π_q^J}¦¨©a#f”MC¢¢^`iutJXCžC¢M]CrkaX\–JKNKrkieb—ªCM ZeGJM#¥Yb“X\b“^`a/^kfÀc¿¤¼©Á½¤·gÂk½HÀ,½¤·gÂk½u¿ YVZeGJM]uGJ^8b“]LM

`¦H^kfxr@aJ^`aJb“aWf”^8i\Kr‰Zeb—¥8Mmsci\b“^`i

π^{J b“a} Π^JÍ

~W¦H^kfxrka/MLKNsJb“i\b’]Lr`§^`i b“Krk_`b“acr`i\VŸ{}F&TX\scr`]LM

χžctWMCs>MCactWb“aJ_^`a£tJr‰Zur

X^nÍ

J¦H^kfxr@K@M‘r`X\–JieM

gi

^8a χ^¦ aYVŸMC—MCK@MCaœZ&^kf

χ ¢ b““ >M]Crk“—M‘t

X(l)b“aZeGJMf”^`““^‰¢ b—aJ_>¦

Œ MtWM‘]b’tWM^kf$Z\GJM#sJie^`s,MLi

π_i^{M IA} ]CrkactJb“tJrkZ\M#X\–c]uG/r8XZ\GJM#KNM‘rka/¢ bdZeG™i\M‘X[s,MC]ZZe^

gi

π^{M IA}_i (θ) ∝ Z

χ

π^J(θ|X(l))gi(dX(l)). ^¯"

FHGcMLažWf”ie^`K ^`aJM

π^JžWKr`aœV{™¨ ]Lr`actWb’tJr‰ZeMCX ]Lr`a ,M#ML“b’]b—Z\MCt¦

Œ M„sJie^`s,^8X\MNb—a®ÏML¡caJb—Z\b“^`a²~/Ze^}]^`a>X[b’tWMLiZ\GJM´º‰½¤·¹¼Y¶Á¼6· `º‰¼©º‰Ó[ºC¾k½eÁeÁL¶Á¸>¼ L½¤·¹¼©Á½¤·gºk¦™¨©aZ\GJM

f”^8—“^‰¢ b“aJ_^kfxZeGJb“X±rki\Z\b’]“M`ž>¢M@]^8K@s>rkieMZ\GJM(aœ–cK@MCi\b’]Lr`ieMCX\–J—ZeX&^`f0Z\GJb’XM‘XlZeb—KrkZ\b“^`a™¢ b—Z\G}ZeGJM^8aJMCX

tJMLieb—¥8MCt(f”i\^8K MLÅWsJ—b’]b—Z{™¨ sJieb—^8ieXC¦Î^kZ\MZ\GcrkZtJrkZerr`i\M –cX\MCt@Zl¢ b’]Mb—a„]Li\b—Z\MCi\b“^`a ! $Ze^^‰¥`MCie]L^`KNM

ZeGJM’r`]u›/^kfb“aWf”^8i\Kr‰Zeb—^8a¦FHGJM@X[M‘]^8act™sJie^`s,^8Xerk 6w rksJs,MCr`i\M‘t/Z\^„ ,M“MCXeX±XlZurk J“Mb“a˜X\b—K(–J“rkZ\b“^`acX

¢ GcMLaf”ML¢StJr‰Zur@r`i\M#r¥‰rkb“’rk J“M`¦

#~W¦Á¼

π^J uÁ#º‰¸£·¹¶À,½\ÂuÀJÁL½À,½¤·g‰½±·¹¸

Π^{J ºk¸ (»—ÁL¼} (X(l1), . . . , X(lL)) ^uÁ L ^{ЮÑÒ Â‰½} π^J%NÑ cÁ@ºk½¤·¹¼W¶NÁL¼6· 8º‰¼©º‰Ó[ºC¾k½\ÁuÁL¶NÁL¸c¼ L½¤·¹¼<ÁL½¤·g‰¸*·’¿

CohA(π;Xⁿ) = 1 L

XL

i=1

KL

π^J(.|X(li),Xⁿ) ||π

KL{π^J(.|X(li),Xⁿ) || π^J(.|X(li))}. ^!

»—Áuº ‰ÁÓ[Âk¸,ÁLÓ<вÑÒ,Ó[ÂW¼ ÁL½u¿¤·g‰¸Â L½¤·¹¼©Á½¤·gÂk¸ N·’¿

CeohA(π;Xⁿ) = 1 L

XL

i=1

KL

π^J(.|Xⁿ) || π

KL{π^J(.|Xⁿ) || π^J(.|X(li))}. ^¯w

mÁ¶º‰½ TY^`KNM^kZeGJMLis>^œX\X\b— cb—“bdZeb—M‘XrkieMML¥8^`›8MCt*b—a«ŽHrV`M‘X[b’rka*—b—Z\MCierkZ\–JieMNf”^`iZeGJMŸ–cX\MŸ^`fsJie^`s,MLi

sci\b“^`iuXL¦Ž^Å˜rka>tFHb’rk^CykJž§s¦ê~k @»—Âeºk»¹»W¸>·Â‰½¤¶ sJieb“^`iuXmG>r¥`MtWMLa>X[b—ZlV˜¢ GJb’]uG´X[“^‰¢ —V˜¥r`i\b“MCX

^‰¥8MLiZ\GJM/i\MC_`b“^`ab“a²¢ GJb’]uG«ZeGJM/—b“›`MC—b“GJ^Y^Wt´f”–Jac]Z\b“^`a b“X@]^8ac]MCa8ZeierkZ\M‘t§žrkact²]Lrka® >M/ieML_8r`ietJMCtr`X

_8^Y^Yt„]Lrka>tWb“tcr‰Z\M‘XL¦ r`i[Zeb—_œrka! X&¶º 8·¹¶ W¶ »ý·8Á»d·J‘ #À,½¤·gÂk½u¿ r`i[Zeb—_œrkaž ‘y`y 0Gcr¥8M&_`^Y^WtieMCr`X\^`a>X

Ze^ >M}{}¨ ]Lr`actWb’tJr‰ZeMCXNZe^œ^>¦ ¨<fZ\GJMCb—itWMLacX\b—Z\b“MCXMLÅYb’X[ZCžZ\GJMCV®r`i\M}tWML¡caJM‘t r8XsJieb—^8ieX(f”^8iŸ¢ GJb’]uG

]L^`iei\M‘X[s,^`a>tWb—ac_}ŽrV8MCXMCX[Z\b“KNrkZ\M‘X(r`i\M/r`X\VœKNsWZe^kZeb“]Crk“—VaJML_8—b“_`b“ J“VtWb:MLieMLaœZ@f”i\^8K Z\GJM/Kr‰ÅWb—K(–JK

“b“›`ML“b“GJ^Y^YtMCX[Z\b“Kr‰ZeM`¦

! * 0 . .", * &

Î^kZeM

Π^{M IA}={π^{M IA}_i , i∈I} ⊂Π^I r#X[MLZ^kf§{™¨

]CrkactWb’tJrkZ\MCXxr`act@—MLZ

π^{M IA}_i ∈Π^{M IA ,Mr#{™¨}

]CrkactJb“tJrkZ\M(ML“b“]LbdZeMCt™f”ie^`K

π^J rkact*rŸKNMCr8X[–ci\M

gi

 YV£i\–c—M g ¤¦TY^`KNM@i\–c—M‘X^`fX[MC—M‘]¤Zeb—^8a}b—a

Π^{M IA}

r`i\MZ\^/ >M@_`b“¥`MCa¦#FHGJM@ >M‘XlZ#]Lr`actWb’tJr‰ZeM@X\GJ^`–c“t™ ,M ©ZeGJM@—M‘X\Xmb—aJf”^`ieKNrkZ\b“¥`M@r`X±s>^œX\X\b“ J—M u¦#FHGJM(Zl¢^

f”^8—“^‰¢ b“aJ_(s,^`b“aœZeX ^`f¥Yb“ML¢ KNr`›`MX[MCacX\MmZe^sJieMC]b’X\MmZeGJb’XH¥r`_`–JM#X[ZerkZ\MLKNMCa8Z‘¦

`¦ ¬$b—iuXlZe—V8ž

π^{M IA}(.|Yⁿ) Gcr`XZ\^} ,MZeGJM]L—^œX[M‘XlZtWMCacX\bdZlV^kf

π^J(.|Yⁿ) rkKN^8aJ_}rk“]CrkactWb’tJrkZ\MCX b—a

Π^{M IA}ž0¢ b—Z\G«ieMCX\s,MC]¤Z#Ze^™Z\GcM{™¨ Kr`i\_8b—acr`ptWMCacX[b—ZlV

mi(x) =R

Θπ_i^{M IA}(θ)p(x|θ) dθ^ž

¢ GJMLieM

Yⁿ=Y1, . . . , Yn

r`i\M±b6¦b6¦t¦xier`actW^`K ¥‰rkieb’rk J“MCXtWM¡>aJMCtb—a

Sⁿ=S×. . .×S ^{¢ bdZeG} n

“r`i\_8MmMCaJ^`–c_`G¦ÏmM¡cacMZ\GJM„Á ÀJÁ L¼<Á mÀc¿¤¼©Á½¤·gÂk½½\Á<¾k½eÁ¼r8X

Rⁿ(i) = Z

Sⁿ

KL

π^J(.|Yⁿ)||π^{M IA}_i (.|Yⁿ) mi(Yⁿ)dYⁿ.

¬J^8i(ieMCr8X[^8acX#^kfr8X[VYKNsWZe^kZ\b’]s,^8X[Z\MCi\b“^`iac^`ieKNr`—b—ZlV`ž0–Ja>tWMLiN]’r`XeX[b’]Lr`ieML_`–c“r`i\b—ZlV*]L^`actWb—Z\b“^`a>XLž

Rⁿ(i)→0^{¢ GJMCa} n→ ∞ ¯]¤fl¦O’rkie›`M Cy`y8y" ¦$¨<Z “MCr8tJXZ\^ŸX\ML“MC]Z

π^{M IA} = arg min

i∈I lim

n→∞

X

j∈I,j6=i

Rⁿ(i)

Rⁿ(j). (C1)

~W¦&TYMC]L^`actJ—V8žX\b—KNb““r`i\“V™Z\^}ŽMCi\acr`ietJ^ ‘yky" ¤žtWM¡cacMNZ\GJM˜Á ÀJÁ L¼<Á „¾œº‰·¹¸®·¹¸²·¹¸Â‰½¤¶º‰¼6·g‰¸}sJie^kš

¥Yb“tWM‘t/ œVŸZ\GJMtJrkZercžW¢ GJMLieM±Z\GJM#MLÅYs,MC]ZerkZ\b“^`a£b“XH¢ b—Z\G£ieMCX\s>M‘]¤ZHZ\^

mi(x) Kⁿ(i) =

Z

Sⁿ

KL

π^{M IA}_i (.|Yⁿ)||π^{M IA}_i mi(Yⁿ)dYⁿ

rkact£X\ML“MC]Z

π^{M IA} = arg min

i∈I lim

n→∞

X

j∈I,j6=i

Kⁿ(j)

Kⁿ(i). (C2)

--

1 $1 -1 (L

/2 ¤£

%'12/

¨©a/ZeGJb“XX[M‘]¤Z\b“^`a£rkscsJ—b’]LrkZ\b“^`acXH^`fZeGJMrkscsJi\^œr`]uG/rkieM±sci\^8s>^œX[M‘tŸ^`a£MLÅYs,^`acMLaœZ\b’rkr`act Œ MCb— J–c—KN^WtYš

MC“XC¦&FHGJM‘X[MtJb“X[Z\ieb— c–WZ\b“^`acXmrkieMrkKN^`ac_Z\GJMK@^œXlZ±–cX\MCt/f”^`imKN^YtJML“—b“aJ_ZeGJM“bdf”MLZ\b“KNM^kfrb“actW–>XlZei\b’rk

]L^`KNs,^`aJMCa8Zr`act£MCX[Z\b“Kr‰Zeb—aJ_NZeGJMLb“is>rkiurkKNMZeMLiuXb’X&^`fxsJi\b“KrkieVŸb“a8ZeMLieMCX[Z&b—a™Kr`aœVi\MC—b’rk cb—“bdZlV/XlZe–ctYš

b“MCXC¦±npX[s,MC]Lb“r`—“V™b“a}ZlVYsJb’]Lrk0]Lr8X[M‘X^`fp]MCacX[^8i\M‘t§ž:X[Krk“—š<X\b—ªCMCt}tcr‰Zercž,Z\GJM@–cX\M^kfŽrV8MCX\b“r`a£b—aWf”MCi\MCac]M

b’XNtWMCX\b“ier` J—M/rkact²Z\GcMKN^WtWML““b—aJ_^kfX[–c WjlMC]Z\b“¥`M/sJi\b“^`iNtWb’XlZei\b“ J–WZeb—^8acX@b“XNr`a²b“a8ZeMLacX\b“¥`M£tW^`Krkb“a²^`f

ieMCX\MCr`ie]uG¦$TYMCMf”^`i b“acXlZurkac]LMTYb—ac_`sJ–Jie¢Hrk““r@rkact£{}r`^ Cy"" žJrkact/ŽMLie_`MCiHr`act£TY–Ja‘y`y`cž Cy`y! ¦

+ . . &+. &,*$ ( .

Œ

M£X[–Jscs>^œX[M„Z\^˜G>r¥`M

Xⁿ = (X1, . . . , Xn) tJrkZer˜¢ bdZeG²s:tYf

p(x|θ) = θexp(−θx) 1{x>0}

¦ Π^J

b’X#ieMCX[Z\ieb“]Z\M‘t}Ze^ZeGJM 8M:i\MCVWXsJi\b“^`i

π^J(θ)∝ θ^−1¦ {}F&T˜f”^`iZeGJMŸMÅWs,^`aJMCaœZ\b’rkxKN^WtWML6žb“a´Z\GJM

–cac]MCacX[^8i\M‘t]Lr8X[M8ž$b’X@r˜X\b—aJ_8—M/tJr‰ZurJ¦Î&^kZeM„ZeGcr‰Z(Z\GJM/ieb—_8G8Z\š<]LMLacX\^`ieMCt´]Cr`X\M„]Lr`a²rk—Z\MCi\acrkZ\b“¥`MC—V >M

X\^`“¥`M‘t–cX[b“aJ_}X\s>M‘]b’rk `M ,ieMLVWX#sJieb“^`iuXtWM¡>aJMCt´ YV´ÏM£TWrkaœZ\b’X#MLZ£º‰»% ¯~`z`z ±^`i–cX[b“aJ_¿LÁ cÁ¸>¼¯·gºk»

¶@·¹¸c·¹¶º‰»x¼6½\ºk·¹¸c·¹¸œ¾™¿Lº‰¶À,»—Á¿ 6TW{}F&T ±b“ac]“–ctWb“aJ_™]MCacX\^`ieMCt}^8 cX\MLie¥rkZ\b“^`acXCž§tWM¡>aJMCt* YV˜ŽMLie_`MLirka>t

h0MCi\ieb“]uGcb 6~kz8z8~ ¦x¬J^`i ieMCr8X[^8acX^kfxX[b“KNsJ—b’]b—ZlVŸZeGJb’X ]Lr`X\M¢ b—“aJ^`Z  >M]L^`acX\b“tJMLieMCtGJMLieM`¦

8 ‚ Š`ˆÌ gÌ$ˆW† Ç … Î&^`Z\MZ\GcrkZ™Z\GJM 8M:ieMLVWX/s>^œXlZeMLieb—^8i/b’X/Z\GcM

G(n,Pn

i=1Xi) tWb’X[Z\ieb— J–JZ\b“^`až

¢ GcMLieM

G(a, b) b’XZ\GJM£QrkKNKr}tWb’XlZei\b“ J–WZeb—^8a¢ bdZeG«KNMCrka

a/b r`act´¥‰rkieb“r`ac]M

a/b^{2¦}¬Jie^`K} π^{J ¢M}

sci\^8s>^œX[MmZ\GJieMLM#{™¨

]CrkactJb“tJrkZ\M‘X tWMLs,MLactJb—aJ_Nf”ie^`K tJr‰ZurJ¦

`¦±Ò ·gÁL¸c¼Âk¸`¾œº‰¼©Á½¤·gÂk½ 6Ò ‰¦cOGJ^Y^8X\b“aJ_

g1

r8XZeGJMmÏmb“ier8]tWb’XlZei\b“ J–WZeb—^8a„]^8ac]MCaœZ\iur‰Z\M‘t

r‰Z£Z\GcM´X\–Wß]b“MLaœZ}X[Zer‰Zeb“X[Z\b’]b—a ie–J—M ¯ ¤ž#r acr‰Ze–JiurksJieb—^8ib’X/Z\GJM´MLÅYs,^`acMLaœZ\b’rk]L^`a‰jl–J_œr‰ZeM

π^{M IA}₁ (θ) =G(1, n⁻¹Pn

i=1Xi) ]MCaœZ\MLieMCt£^8a™Z\GcM(Kr‰ÅWb“K–JK “b—›8ML“b—Gc^œ^Wt™M‘XlZeb—KrkZ\^`i g{˜on