Binary Feature Selection with Conditional Mutual Information
20
0
0
Texte intégral
(2) INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE. Binary Feature Selection with Conditional Mutual Information François Fleuret. N° 4941 Octobre 2003. ISSN 0249-6399. ISRN INRIA/RR--4941--FR+ENG. THÈME 3. apport de recherche.
(3)
(4)
(5) !#"$&%& &(') & *+&,-/.01 2 /3 465+78:9+;=<>3@?BA:C4DAFE GIHKJLMN2OQPSR:TUMDVXWFYZTX[]\FRHK\FL^LM_=L`WaY Hb[]RKMFc [dL`WfeFMDgDcihK\aRKRKjDMDgDcKY\FRKRbWf[kgXgUWaRbYM6g lVU\amnMToPSLMDhK[dW poWfqbqr\FVsTohtMVXMDY HKMDVXY HbMuR+v!wFxawby&O{z@YTU\a|bVUM}f~F~aNOyD^qbWaeaMDg. !D -Kaa PSRWY\aR:TUMiTI\a!YdWFgUgU[+YW-TX[]\FRc:M1qKVX\aq+\FgUMTX\bgsM1Y\FRbht[]TU[d\aRbWaL2tTXbWf[dRt\FVUL`W-TX[]\FR TX\gsMD]M6YZTW@WfL[d]\a+|b[]RbWaVU1M6W-TXKVUM6gHK[kY HWaVUM[dRbht[d:[khtbWa]d2ht[kgXYVX[]L[dRbW-TX[]RKeWaRbh2MDWai]2hKMq+MRt_ hKMR:TDMgUHK\- TXHbW-T\aR>W1T WagU\a[]L`WfeFMYkWagXgU[rYDW-TX[]\FRcFhtM6gsqb[TXMo[T ggU[dL^qb][kY[]TnaciWRbWa[]FM¡IW¢aMDgU[kWfR YdWFgUgU[]bMV/|bWFgsM6h2\FR^MDW-TXKVXMDggUMdMDYTUMDh^[]TUH^TUHb[dgI£\aRbht[]TU[d\aRrWfK¤¥tTXbWfrPSRt\aVXL`W-TX[]\FR^¤¦Wft[]L[d§DW-TX[]\FR ¨ £¤¥Pn¤ª©YVX[TXMVX[]\FR¦q+MVU\aVXLgWagMD]Wag@W&YkWagXgs[]bMV|KK[dT1[]TUH¬«hbWa¡\i\FgsTD@¬MWfkgs\gsHK\-TXHbW-T TXHK[kgoYkWagXgs[]rYDW-TU[d\aRLMTUHb\ih&[kgIL\aVXM@VX\a|bbgnTITXHbWfR|+\:\:gnTX[]Rbe^HbMRTUV Wf[dRKMDh\aR¥W^RK\a[kgUhKW-T WgsMTD ®¥¯:°²±X³2´ -µ!a YdWFgUgU[]rYWfTU[d\aRciM6W-TUbVUMgUMdMDYZTX[]\FRcK¡IW¢aM6gs[kWfRYdWFgUgU[]bMV6ctL2tTXbWf[dRt\FVUL`W-TX[]\FR. Unité de recherche INRIA Rocquencourt Domaine de Voluceau, Rocquencourt, BP 105, 78153 Le Chesnay Cedex (France) Téléphone : +33 1 39 63 55 11 — Télécopie : +33 1 39 63 53 30.
(6) ! % % & .0 >.0 1 %
(7) s&,-.0 . >! &%& && WaRbgKRªY\FR:TUMiTXM2htMYdWFgUgU[]rYWfTU[d\aRcKRb\abgqKVX\aq+\FgU\aRbgoh tTX[]d[kgsMDV [dRt\aVXL`W-TX[]\FR¥L2t_ TXKMddM^Y\aRbht[]TU[d\aRbRKMd]Mq+\aKV1gUjdMDYZTX[]\FRKRKMDV@bRKM2WaL[]d]M`htMhtM6gUYVU[dqtTUMDKV g|K[dRbWa[]VXMD g :K[/gs\FR:T@[dRbht[di[_ hKKMd]MDLMR:T[dRt\aVXL`W-TX[gTU\FtTMDR¦jT WfR:ToWf[d|KdMLMR:Thtjq+MRrhKWfR:TXgMR:TUVXM2MDt o\FbgL^\FR:TUVX\aRbgogUKV bRqKVX\a|b]JDL^M1htM1YkWagXgs[]rYDW-TU[d\aRh [dL`WfeFMDg :KML`Wa]eFVUj1gUWgs[dLqKd[dY[TXj@bR&YdWFgUgU[rMKVIhtM@TniqrM¡IW¢aj_ gU[dMRRb W BtTX[]d[kgUWaRFThtM6ghtMDgXYVX[dqtTUMDKVXg1gUjdMDYTU[d\aRKRKj6ghtM`YMTsTUM`L`WaRK[]JDVUM^\a|KTU[dMR:ThtMDg@T Wft¬h MDVUVXMKV gU[dL^[dkWf[dVUM6 g YMKªh bRYdWFgUgU[rMKV1Y\FRbgnTXVUb[ T Wa[dhKM^h «hKWF¡\i\:gnT6 \abg@L\aR:TXVU\FRbgjDeFWa]MDL^MDR:T :KMYMTsTXM1TUM6Y HKRK[ FbMMDgsTI|+MDWabY\FKq&qK]rgVU\F|KbgsTU!M :KM1]M|+\i\FgsTU[dRKe`hKWfRrgIKR¥YWahKVUM1|KVXK[]TUja "$´ 6 ±
(8) # ` YdWFgUgU[]rYWfTU[d\aRcigsjD]M6YZTX[]\FRhtM@MDWfTUKVXMDgDctYkWagXgs[]bMDKV|rW¢ajDgU[dMRc:[]RK\aVXLWfTU[d\aRL2tTXKMd]M.
(9)
(10)
(11) . N. +* `%& ¡`VXMDhtrY[dRKeTXHKM1R:bL2|+MVI\f Y\aRbgU[khtMVXMDhM6W-TXKVUM6gci\FRKM@YDWfR|r\aTUH&VUM6htbYM\-aMDVs_=KTsTX[]Rbe2\f ]M6WfVXRK[]Rbe LMTUHK\thKgDcbWfRrh[dRbYVXMDWFgsMTUHbMY\aLqKKTXW-TX[]\FR¥gsq+MM6h\a!qKVXMDhK[dYTU[d\a R ! ~F#N " /¬M1\tYbg[dRTXHK[kgIqbWfq+MV \FRTXHKMgsMD]M6YZTX[]\FR\a!WM TUMRrgI\f |b[]RbWaVU>MDW-TXKVXMDgIWfL\aRbe^WgUMT\fgsMDaMV WfrTUMDRbgI\f!TUHb\abgXWfRbhbg\a TXHKML []R¥WY\FR:TUMiT\f/YkWagXgU[rYDW-TX[]\FR GIHKML\FgsTgnT WfRbhbWfV h W¢tg1TX\ªgsMD]M6YZTMDW-TXKVXMDg2Y\aRbgU[kgnT2[dR V WfRKi[dRKe¥TUHbML WaYDY\aV ht[dRKeTU\¦TUHKMD[]V [dRbht[di[dhKbWfqKVXMDhK[dYTU[daM&qr\-MV6cHK[kY H [kgMDgsTU[dL`W-TUM6h |i -WfVX[d\abg^LMTXHK\thKg`gsrY H Wa%g $![kgsHKMDV`gUY\aVXM $!£ ~a#~ "=' c &\adL\aea\FVU\-:)_ (iL[]VXRK\-ªTUMDgsTDclM6WfV gs\FR Y\aVXVUMDdWfTU[d\a*R ¤¥l~a#~ "@\aV>LtTUrWf[dRt\FVUL`W-TX[]\FR ¡IW-T xfwtc¡ xatcG \aV ~a+N " , (tMdMDYZTX[]\FRª|bWFgsM6h¦\FR¬gUbY H¬WVXWaRKi[]Rbeht\iMDg1Rb\fT1MRbgUKVXM^Wea\i\th¦\FVsTXHK\aea_ \FRbWfd[]Tn |+MTnMMDR M6W-TUbVUM6gc@WfRbh YWfR ]M6Wah TX\ VUM6htKRbhKWaR:T&WfRrh TUHibg&dMDgXg&[]Rt\FVUL`WfTU[daM¬gUMdMDYTUM6h WaL[]d[]M6g zKVWfqKqKVX\FWFY HY\aRbgU[dgsTXg[dR`qK[kY i[]RKe@M6W-TUbVUM6g/HK[kY H^L`Wfi[dL[]§DMITUHKMD[]V L2tTXbWfK[dRt\FVUL`W-TX[]\FR[]TUH TXHKMoYkWagXgTX\qKVUM6ht[kYZTDc:Y\FRbht[]TU[d\aRbWa]dTU\1TXHKMoVXMDgUqr\FRbgUM\a²WaRiMDWfTUKVXMoWfdVUM6Waht2qK[dY FMDh!GIHK[kg£\aRbhK[_ TX[]\FRbWf ¤¥tTUrWf PSRK\aVXLWfTU[d\aR¥¤¦Wfi[dL[]§6W-TX[]\FRYVU[]TUMDVU[d\aR ¨ £¤¦Pn¤ª©oht\iM6gRK\fT@gsMD]M6YZToM6W-TUbVUM6gogs[dL[]kWfV TX\@Wa]VXMDWFht1qK[kY aMDh2\aRKM6gc-MDaMDR2[]bTUHbM2WaVUM[]RbhK[]i[khtbWfddqr\-MVUK=cfWag TUHKMD2ht\@RK\fTYDWfVXVUWahbht[TX[]\FRbWf [dRt\FVUL`W-TX[]\FRWa|r\FtT/TUHKMoYdWFgUg TX\@qbVUM6ht[dYTDGIHibgcF[T/MDRbgsbVUM6g/WeF\i\ih2TUV WahKM
(12) \ -`|+MTnMMDR^VXMDhKKRbhKWaRbY WaRbh&ht[dgXYVX[dL^[dRbWfTU[d\aR /tq+MVX[]LMDRFT go\FRW^WFYMtgD RK\FRt_BWFYMYdWFgUgU[+YW-TX[]\FRT WagUhtMDL\aRbgsTUV W-TXMTXHbW-TMDW-TXKVXMDgoY HK\:gsMDR WFYY\aV ht[]RbeTX\TUHb[dg!YVX[TXMVX[]\FRYWfR|+M M .>Y[dMR:TUd@Y\aL|K[]RbMDh1[]TUH2WRrWf[daM ¡IW¢aM6gs[kWfR1WfqbqKVU\:WaY ,H !/!0 NKc 1 PnGxF+} "@WfRbh dMDWFh TX\(MVXVX\aV`V W-TUM6g`gs[dL[]kWfV>TU\ TXHK\FgUM¥\a|tT Wf[dRKMDh []TUH «hbWa¡\i\Fgs2 T $3(txaF4W " «dgU\bc MtqrMDVU[dLMR:TXggUHK\-$TUHKM>VX\a|bbgnTXRKMDgXg1\fTUHb[dg2L^MTUHK\th []TUH VXMDgUqrM6YZTTX\¥RK\a[kgsªTUV Wf[dRK[dRKe¥gsMTXgDc!WFg[T WFY HK[dMaM6g|+MTsTXMVVXMDgUKT gTXHbWfRVXMeFKkWfVX[]§DMDh1«hKWa¡\i\FgsTDc¢MDaMR1TXHK\abeaH[]ThK\:M6gRK\aT VXM :K[dVUMTUHKMTUKRb[]RKe \a!W^VXMeabdWaVU[d§DWfTU[d\aRqrWfV WfLMTXMV6 PS6R 5Z}^MgUKLL`WfVX[]§DM1TUHKM2Y\aR:TUMiTo\fTUHK[kg\aVX²cKM[dR:TUVX\thtbYM1TXHKMRK\aTXWfTU[d\aRbgI\FVTUHbMVUM6gnTo\a TXHKMqbWfq+MVWaRbh¦Mea[daMWgsHK\FVsTgUKLLWaVU¥\agsTXWaRbhKWfV hªYdWFgUgU[]rYWfTU[d\aR¥TXMDY HKRK[ :KMDgD^¬M :K[kY i] qbVUM6gsMDRFTTU\i\akgVX\aL []RK\aVXLWfTU[d\aRTXHKM\FVUWfRbh¥hKMDgXYVX[]|+M2\FKVoM6W-TUbVUM_ gUMdMDYTU[d\aR¥gUY HbMLM[d7R 5XNbo¬M eF[]FM`VUM6gsbT g\foMtqrMDVU[dLMR:TXg 8¬L`Wf[dRK] Y\aLqbWfVX[kgs\FRbg1[TXH «hKWF¡\i\:gnT 8[]9R 5XwªWfRr:h 5;tc/WaRbh M rRbWfd]ht[dgXYrgUgTXHK\FgUM1VUM6gsK]TXgWaRbhTUHKM1\FVsTXHbY\FL[]RKehtMDaMd\aqbL^MDR:TXgI\f TUHK[kg\aVX`[] R 5 K lM\FqKdMo[]RWHiKVUVX2Hb\2WfVXMIWfL[d][kWfV[TXHTUHKMTU\aqb[dYoYDWfR1mnKLqhK[]VXMDYTUd2TX\2gsM6YZTX[]\F2R 5 }tdy ¨ qbWfeFM N:©\FVTXHKM@RK\fT W-TU[d\aRrg<c 5XNb } ¨ qbWaeaM 0 ©\FV TXHKMhtMDgXYVX[]qKTU[d\aR>\aTUHKM£¤¥Pn¤ Wa]eF\aVX[TXHKLc:WaRbh`TX\T Wf|KdMDg yWaRbh&} ¨ qbWfeFMDg@yayWaRbhªyDw:© \aVMDVUVX\aVIV W-TXMDgIWfRbhY\aLqbWfVX[kgs\FR[]TUH¥«hKWa¡\i\FgsTD "$ ? > = . +. @BADC. E*F3GHGJI)FLKNM. ¡ M6YWfrgsMTUHK[kg\aVXIWag\aVX[dea[dRbWfd]L\aTU[d¢WfTUM6h|iTXHKMohtM6gs[deaR\f²W@WaYMIhKMTUM6YZTX\aV6cf\aKV/MtqrMDVU[dLMR:TXg WaVUM|bWagUMDh \aR W¬YdWFgUgU[+YW-TX[]\FR(\fWaYM&tg RK\aRt_=WaYM[dL`WfeaM6gO$b\aV^TUHrW-T`VUM6WagU\aRcM[dR:TUVX\thtbYM Rb\fTXWfTU[d\aRbgoVUMDdWfTUMDhTU\[]L`WfeFMYkWagXgs[]rYDW-TU[d\aR\aVoTUHKM2gXWfFM1\fKRbhtMDVXgsTXWaRbht[dRKebcr|KtT\FKVoWfqKqbVU\:WaY H&[dg eFMRKMDVU[kYWfRrh¥YWfR|+MWaqKqKd[]M6hTX\WfRigsMT\f/|K[dRbWaVUMDW-TXKVXMDgDM2eF[]FMHKMVXMW`VX\aKeFHhKMDgXYVX[]qtTX[]\FR \a TXHKMMtq+MVX[]LMR:T WfgUMTsTX[]RbeFg ¨ [dL`WfeFMDgDciMDW-TXKVXMDgDctMTXYa ©^|btToM1eF\^[dR:TU\`htMTXWf[dkgI[] R 5Uwr PQPSR#TJU VU4W.
(13) w. . 1 MT htMDRK\fTXM2TXHKMgsMT1\a qK[]tMeaV W¢tgUYDWfdM[dLWaeaM6gcWfRrh WVXWaRbht\FL -WaVU[kWf|KdM2\FR gsTXWaRbht[dRKe¥I \FVTUHKM&ht[kgnTXVU[d|KtTX[]\F28R \f×[]28L`WaeaMDgD¬M&htMRb\fTUM Y W¥|+\:\F]M6WfXR V WfRbht\FL -WaVU[kWf|KdM\aV2TUHKIM YdWFgUgTX\`qKVUM6ht[kYZTDciTXHKM-WfdKM 0 gnT WfRbhK[]RKe^\FV # WaRbh 1 \FV ¨ YnB5Uwr]y¢©Z MY\aRbgU[dhKMVW`gUMT@\f/|r\i\F]M6WfR&MDWfTUKVXMDg cbHK[kY H¦WfVXML`WfqKqb[]RKe:gVX\aL GIHbMRiKL|rMDV^\aoMDWfTUKVXMDg N [dg^qKVXMTUTn dWaVUeFfMac,\f.. .TU,HKfM&\aV htMV^\fTUMRrg\aTXHK\abgXWfRrhKg IPS→R TU{0, HKM&M1}i_ q+MVX[dL^MDR:TXg1TXHK\FgUMMDWfTUKVXMDgWaVUM|+\i\adMDWfRKRrYZTU[d\aRrgHK\:gsM`-WfdKM6ghtMq+MRrh [TXH¬TXHKM>qKVXMDgUMRrYM`\aV Wa|bgUMRbYM\aMDhKeaMDgWfT`YMVUTXWa[]R qr\:gs[]TU[d\aR []R TUHbM¥qK[dYTUKVXMa: WFY H \fTUHKMDL [kg^TUHibg`htMbRKM6h |i([]TXg d\tYWfTU[d\aR&[]RTUHKM 28 × 28 qK[kYZTXKVXMV WfLMFct[T g\aVX[]MDR:TXW-TX[]\FRWaRbh&[T gTU\adMV WfRrYM ¨ Yn 5Uwb }a© MhtMRK\aTUM TUHbM@|+\i\adMDWfRVXWaRbht\FL -WaVU[kWf|KdMDgIgsTXWaRbht[dRKe2\FVTUHKM VXMDgUq+\aRbgUMDg \fTUHbFMM6=W-TUfbVU(X), M6g!¬. M@. ., F[d]²\f= TXMfR>L`(X)WaaMWY\FRtbgU[]\FR>|rMTnMDMR`TXHKMMDW-TXKVXMDgWagLWaqKqK[dRKeFg WaRbh&WagIV WfRbht\FL -WfVX[dWa|KdMDg ¨ [= MF/|rMTnMDMR&TUHbM f WfRbhTUHbM F ©Z $![dRbWfd]FcMhKMRK\aTUM TXHKMgUMdMDYTUM6h MDWfTUKVXMDgDcoHKMDVUM [dgTXHKM¬RiKL|rMDV\a M6W-TXKVUM6gIMgUMdMDYTDcKWfRrh[kfgIaMDVU,. .gs.L`, fWfdY\FLqbWfVXMDh>TU\ N ¨ \f!TUHbM\aV htMVI\f KMTnK©Z «]@gnT W-TU[kgsTU[kYWfM6gnTX[]L`WfTU[d\aRbg>hKKVU[dRKe TXVXWa[]RK[dRKe WfRrh TXMDgsTU[dRKe(WfVXM|bWagUMDh \FR gXWfLqKdMDg`\a1WMD HiKRrhtVUM6hKg\f²qK[kYZTXKVUM6gcadWa|rMD]M6h|iHrWfRbh^[]TUH^TUHKMD[]V VUM6WfKYkWagXgUMDg (x , y ), . . . , (x , y ) ¨ YZn 5Xwbdy6©Z GIHb\FgUMgUMTXgY\FR:TXWf[dRWFg L`WfRiWaYM@WaRbh`RK\FRt_BWFYMqK[dYTUKVXMDgDciHK[kY HY\FVUVXMDgUqr\FRbh^TU\^WfR>M :K[]d[d|KVXWfTUM6h qbVU[d\aV P (Y = 1) ' 1/2M[]dbgUMTXHKML []LqKd[kY[]TUd\FVWfdTXHKMMLqK[dVX[dYDWfMDgsTU[dL`W-TX[]\FRbgohtKVX[dRKe TXVXWa[]Rb[]RKe\FVTXMDgsTU[dRKeb 1. 1. 1. N. N. N. i. i. n(M ). n(1). 1. @BA@. 'G JFLK. 1. T. T. H3K
(14) K H3I6H M IDH KDH3M M I. @[daMDR^W@gUK|bgUMT\frMDWfTUKVXMDg WfdVXMDWahKgsMD]M6YZTUM6hcfMY\aRbgU[khtMV/d[]RbMDWfV/htM6Y[kgs[d\aRVUb]M6gc Hb[dY HhtMq+MRrh\FR&TUHKMgU[]eFRf\a!WaR&, .M.t.qK, VUfM6gUgU[d\aR\a TXHKM1\aVXL n(M ). n(1). f (x). =. M X. ωi fn(i) (x) + b. M2HbW¢FMbgUMDh&Tn\>Wfdea\aVX[]TUHKL`gTU\MDgsTU[dLWfTUMTXHKM WfRbh VX\aL TUHKM2TUV Wf[dRK[]Rbe>gsMTD GIHbMrVXgsT^\FRKM[dgTXHKM¦YkWagXgs[kYWaq+MV YMDqtTUVX\aR* po\Fg; ic (ωo\-,t.:.}+."o, ωWfRbh ) TUHKM¥gUbMDY\aRbh \aRKM[kg^TUHKMRbWa[]FM ¡IW¢FMDgU[dWaRYkWagXgs[]bMD!V !/!0 Nbc 1PnGx:+} " ¯ ¢ ¯ D ! ´ PSR WRitT gsHKMD]=c TUHKM`q+MV YMDqtTUVX\aR dMDWfVXRK[dRKegUY HKMDLM>Y\aRrgs[kgnT g1[]R M6gnTX[]L`WfTU[dRKe¥[TXMV W-TX[]FMd¥TUHKM`FMDYTU\aV |i>Y\aVXVUM6YZTU[dRKe[ToWagd\aRKeWagTXVXWa[]RK[dRKe^MKWfLqKdMDgIWaVUML^[kgXYkWagXgs[]bMDh¤¦\aVXMqKVXMDY[dgUMdac (ω , . . . , ω ) WFg]\FRKe Wag^TXHKMVXM¥Mt[kgnT g`W L^[kgXYkWagXgs[]bMDh MKWaL^qb]M (y , x ) [T gMDWfTUKVXM¥aM6YZTU\FV[dg>WFhKhtM6h TX\ TUHKM Rb\aVXLWa²aMDYTU\FV[[To[kgI\f/YkWagXgqr\:gs[]TU[daMFcKWfRbh&[kgI\fTXHKMVX[dgUMgsb|tTUV WaYTUMDh i=1. 1. 1. M. M. i. ∀k. ωk. i. ← ωk + (2 yi − 1) fn(k) (xi ). IG HKM|b[dWFg TXMVXL [dgY\aLqKtTXMDh&|iY\FRbgs[khtMDVU[dRKeWY\aRbgsTXWaRFTIM6W-TUbVUMWfdIW¢igM:bWa²TU\ P !TUHKM TXVXWa[]Rb[]RKe&gsMT@[kg@][dRKM6bWfVX]¥gUMqbWaVXWa|K]M[]RªTUHKM^M6W-TUbVUM^gsqbWFYMFc²TUHKM^qKVX\iYMDgXg[kg@qKVX\-aMDh&TX\&Y\aRi1aMDVUeFMTU\ "$#P"&%.
(15)
(16)
(17) ;. WgsMDqbWfV W-TX[]RbeHiiqrMDVUqKkWfRbMWfRrh¥TUHbMRiKL|rMDV@\a []TUMV W-TX[]\FRbgYDWfR¬|rM^MDWFgs[d]¥|+\abRbhtMDh ¨ gUMM £ (KG~a4~ " qrWfeaM6gy6}¢_ ywi©ZP rTUHKMohbW-TXW@WfVXMIRK\fTgUMqbWaVXWa|KdMac6TUHKMqbVU\tYM6gUg!RKMDaMDV!MDRbhKgDcaWfRrh2HbWFg!TU\1|+MTXMVXL[]RbWfTUM6h Wf TUMDVoWRiKL|rMDV\f[TXMV W-TX[]\FRbgKtMDh&W^qKVU[d\aVX[= ¯ °r¯ GIHbM2RbWa[]FM¡W¢FMDgU[dWaR¥YkWagXgs[]bMV[dg@W>gU[]LqKdM2d[]FMd[]Hb\:\thV W-TX[]\`TXMDgsT[]TUHªWaRªWagXgUKLqtTU[d\aR¥\a Y\aRrht[_ TX[]\FRbWf[dRbhtMDqrMDRbhtMDRbYM|+MTnMMDRTXHKM1MDWfTUKVXMDgD/GIHKM1qKVXMDht[kYZTXMDhYkWagXgIhtMq+MRbhbg\aR&TUHbMgs[deaR&\f f (x). = ln. Pˆ (Y = 1 | Fn(1) = fn(1) (x), . . . , Fn(K) = fn(K) (x)) Pˆ (Y = 0 | Fn(1) = fn(1) (x), . . . , Fn(K) = fn(K) (x)). RbhtMVoTUHKMWFgUgUKLqtTU[d\aRTUHrW-TTUHbM . Fn(.). WfVXMY\FRbht[]TU[d\aRbWa]d>[dRbhtMq+MRrhtMR:TDcbea[daMR Y cKM1HbW¢aM . Pˆ (Fn(1) = fn(1) (x), . . . , Fn(K) = fn(K) (x) | Y = 1) + a Pˆ (Fn(1) = fn(1) (x), . . . , Fn(K) = fn(K) (x) | Y = 0) QK ˆ i=1 P (Fn(i) = fn(i) (x) | Y = 1) + a = ln QK Pˆ (Fn(i) = fn(i) (x) | Y = 0). f (x) = ln. i=1. =. X. ln. i. = =. Pˆ (Fn(i) = fn(i) (x) | Y = 1) + a Pˆ (Fn(i) = fn(i) (x) | Y = 0). (. Pˆ (Fn(i) = 1 | Y = 1) Pˆ (Fn(i) = 0 | Y = 1) ln fn(i) (x) + ln (1 − fn(i) (x)) Pˆ (Fn(i) = 1 | Y = 0) Pˆ (Fn(i) = 0 | Y = 0) i ( ) X Pˆ (Fn(i) = 1 | Y = 1) Pˆ (Fn(i) = 0 | Y = 0) ln fn(i) (x) + b Pˆ (Fn(i) = 1 | Y = 0) Pˆ (Fn(i) = 0 | Y = 1) X. ). + a. i. GIHibgM@bRbWa]d>\a|KTXWf[dRW^gU[dL^qb]MMiqbVUM6gUgU[]\FR>\FVITUHKMY\:M.>Y[dMR:TXg ωi. = ln. Pˆ (Fn(i) = 1 | Y = 1) Pˆ (Fn(i) = 0 | Y = 0) Pˆ (Fn(i) = 1 | Y = 0) Pˆ (Fn(i) = 0 | Y = 1). GIHKMI|K[kWag [kgVXMkW-TUM6h1TU\TUHbMqbVU[d\aV WaRbh2YDWfR|rMMDgsTU[dL`W-TUM6h2eF[]FMRTXHKM ω TX\L[dRK[]L[d§M TXHKMMVXVX\aVVXWfTUbM1\aRTUHKM1TXVXWa[]Rb[]RKe`gUMT6 P (Y = 1) ¯ (t GIHb\FgUM`Tn\YkWagXgs[]bMDVXgHrW¢aMht[ -²MVXMR:T^gsTUVXMRKeaTUHbg^WfRrh M6WfiRKM6gUgUMDgD¦GIHKM&qrMDVXYMqtTXVU\FR TXWfFMDg2[]R:TU\ WFYY\aKR:TTUHKMImn\F[]R:TgsTXW-TX[dgsTU[kYgI\a TXHKM1MDWfTUKVXMDgDcKWfRbhhtM6Wfkg[TXH&\FtTUd[dMV gI|:[]RbYVUM6WagU[]Rbe2TXHKM[dVo[dR
(18) bt_ MDRbYM[]RªTUHKM2bRbWa \FVsTXHK\aeF\aRbWaaM6YZTU\FVDzR¥TUHbM^Y\aR:TUV WfVXacbTUHKM^RbWf[daM¡W¢FMDgU[dWaR¥YkWagXgs[]bMV[dg@|bWFgsM6h i. PQPSR#TJU VU4W.
(19) . . \FRM6gnTX[]L`W-TXMDg\f TUHKMY\FRbht[]TU[d\aRbWa²LWaVUeF[]RrWfbqbVU\F|bWf|K[dd[TX[]M6gWaRbh>[kgL\aVXMVX\a|KrgnTTU\TUHKM1qKVXMDgUMRrYM\a \FtTUd[dMV gGIHK[kg1LMDWaRbgTUHbWfT@HK[ddMTXHKMqrMDVXYMqtTXVU\FR¦[kg@Wa|KdMTX\&hKMDWf[]TUH htMq+MRrhtMR:T1MDWfTUKVXMDgDc[T gU -²MV gIL\aVXMVX\aL \-aMVUKTUTU[dRKe2TXHbWfR&TXHKMRbWf[daM1¡IW¢aM6gs[kWfR @BA. .
(20) 'H 6F FLM G. IG HbM2[khtMDW\f|+\:\:gnTX[]Rbe[kgoTX\&gUMdMDYT@WfRrh¦Y\FL2|K[dRKMgUMFMV Wf YdWFgUgU[]bMV g ¨ \f TUMDRªVXMMDVUVXMDh&TX\Wag
(21) ? crWagoTUHKMD&WFY HK[dMaM1[dRbht[di[dhtrWfd]&HK[deaH¥MVXVX\aVV W-TUM¢©[]R:TX\WL\aVXMead\a|bWa\aRKM[]TUH¦W>a\fTX[]Rbe qbVU\tYM6htKVXMaIPSR¥\abVYWFgsMFcKTXHKMMDWfTUKVXMDgWfVXMY\FRbgs[khtMDVUM6h¥WagMDWaF_ dMDWfVXRKMDVXgDGIHibgDcbTXHKMTXVXWa[]RK[dRKe>\a TXHKM^M6Wf:_=dMDWaVURbMVoY\FRbgU[dgsTXg@gs[dLqKd¥[]R¬qK[dY i[dRKe>TXHKMMDW-TXKVXMDgo[]TUH¬TUHbML[dRK[]LKL MVXVX\aVoV W-TXMa1¬M WadgU\Y\aRrgs[khtMVI\FVMDWFY HM6W-TUbVUM1[]TXgWfR:TX[_=MDWfTUKVXM ¨ [= MFTUHKM\FRKM1HK[dY HVXMDgUqr\FRbhKgTUHbM\aqKq+\FgU[TXM6© GIHKM gUMdMDYZTX[]\FR [kg&hK\aRKM¬|: L`Wa[]R:TXWa[]Rb[]RKe W ht[kgsTUVX[]|KKTU[d\aR \FR TUHbMªTXVXWa[]RK[dRKe MKWfLqKdMDgHK[kY H WFYYKL2KkW-TXMDgI\FRTUHKML[kgUYdWFgUgU[]bMDh&\aRbMDgohtbVU[dRKeTUHbMTUV Wf[dRK[dRKeb«T[]TUMV W-TX[]\FR k cbTXHKMMDWf]M6WfVXRKMV HK[kY H¦L[dRK[]L[d§M6gTXHKM2M[deaH:TXMDh¥MDVUVX\aVoV W-TXM[kggUMdMDYTUMDhc²WaRbhTXHKM^hK[dgsTUVX[]|btTU[d\aR¦[kgVUMVUM6gsHKM6h fTX\[dRbYVUM6WagUMoTUHbMM[deaH:TI\fTXHKML[kgUYdWFgUgU[]bMDhgUWaL^qb]M6gWaRbh>VXMDhtrYMTUHKM@[]Lq+\aVUTXWaRbYM@\fTXHKM1\fTXHKMV g \fTXMTUHbWfT|+\:\:gnTX[]Rbe>YWfR|+MgUMMDRWFgoWKRrYZTU[d\aRrWfeFVXWFht[]MDR:ThtMDgXYMDRFT ¡VUM6~a~tc+¤¦¡I¡ $!~F~i<c $ / G~F#~ " [dRHK[kY HM6WaY HWahKhtM6hMDWa>dMDWfVXRKMDV[kgW`gsTUMq&[dRTXHKMgUqbWaYM@\a!YdWFgUgU[rMV g PSR\FKV Y\aLqbWaVU[kgs\FRbgDcM HbW¢FM/bgsM6h1TUHbM \aVX[dea[dRbWfa«hbWa¡\i\FgsT qKVX\iYMDhtbVUBM $3(txaFWt+c $3(txFf| " c¢HK[kY H [kg@iRK\-R¦TX\gU -²MV@VX\aL \-aMDVsbTsTU[dRKer $K\FV@Rb\a[kgsT WagUigDcMHbW¢FM^Y Hb\FgUMRªW&gs\a Ts_ L`WfVXea[dRªaMDVXgU[d\aR YDWfd]M6h¬«hbWa¡\i\FgsT poz@¤¦x +"=P TVXMeFKdWaVU[d§M6goTXHKM>YkWagXgU[dYDWf/«hKWa¡\i\FgsT1|:ªqrMDRbWfd[]§D[]RbegXWfLqKdMDg Hb[dY H&[]R bKMRrYM1TU\i\HKM6W¢:[dd`TUHKM1TXVXWa[]Rb[]RKercKWagTXHKMWaVUM1bgUbWa]d>\aKTUd[]MDVXgD G \`bgUM|r\i\:gnTX[]RKe`WFgoW^MDWfTUKVXMgsMD]M6YZTX\aV6cKMVXKR&TUHbM2Wfdea\FVU[]TUHKL TU\>WaYDYKLKkW-TUM1TXHKMMiq+MDYTUM6h RiKL|rMDV\aMDWfTUKVXMDgDc:WaRbh`MTUHbMRVXMDY\aLqKtTXMTUHbMM[deaH:TU[dRKe:g[]TUHWaRK\fTXHKMVqKVU\tYMDhtKVXM ¨ qrMDVXYMqt_ TXVU\FR\FVRbWf[daM1¡IW¢aM6gs[kWfRr© 1! & % &%& & . & &,- .01 2 . . n(k). reg. . . BADC. K F HGJI)FLKG F GF FDM.
(22). PSRK\aVXLWfTU[d\aRTXHKM\FVUqKVX\-i[dhtM6g[dR:TUK[]TU[daMTU\i\akg TU\ :bWaR:TU[]^Hb\- L2bY H>[dRt\FVUL`W-TX[]\FR>[kgVXM:K[dVXMDh^TU\ hKMDgXYVX[]|+MVXWaRbht\aL :bWaR:TU[]TU[dMDgDc \aV1HK\-L2bY H[dRt\aVXL`W-TU[d\aR[kggsHbWaVUM6hª|iªWMD$\fTUHbML £GxKy "= MY\FRbgU[dhtMDVIHKMVXM1\aRKd`bRK[]TUM2VXWaRbht\aL -WaVU[kWf|KdMDgIWaRbh&MhtMDRK\fTXM U c V WfRbh W TXHKVUMDM\f TUHKMDL& GIHKM2L^\:gnTbRbhKWfLMDRFT Wf!Y\aRrYMqKT[]R[dRt\aVXL`W-TX[]\FRTXHKM\FVU[dgoTUHKM2MR:TUVX\aqi ) \fWV WfRbhK\aL -WaVU[kWf|KdMa!p\abeaHKdac-[]T FrWfR:TU[]bMDg TXHKMW¢FMV WfeFMRiKL2|+MV\f+|K[]TXgVXM :K[dVUM6h1TU\@MRbYH(U \thtMI\aVhtMDgXYVX[]|+MTUHKM -Wa]KM\a U !« htMTUMDVUL[dRK[dgsTU[kY-WfVX[kWf|KdMHrWagW1RiKdKMR:TXVU\Fq: ¨ WagRb\fTUHb[]RKe[kg/VXM :K[dVUM6hTX\htMDgXYVX[d|rM[]TXg -Wa]KMFcaHK[kY H`[kg/iRK\-R|:WahK¢WaRbYM¢©ZcFHK[]dMW1KRb[\FVUL ht[kgnTXVU[d|KtTX[]\FR`\aR HbWag WfRMR:TUVX\aqi \a l ¬P Wªht[kgsTUVX[]|KKTU[d\aR([kg2bR:|rWfkWfRbYMDhc/[]T^[kgL^\FVUM M .>Y[dMR:T2TX\¬rgsM]M61,gUg2.|K.[.T g, 2TU\¬Y\thtM>VXM :KMDRFT -Wa]KM6gctMDaMDR[] TXHbW-T\FVXYM@TU\bgUML^\FVUM@|K[]TXgI\FVVXWaVUM@-WfdKMDgD/P TYWaR|+MgUHK\-RTXHbW-T l. H(U ) = −. X. log2 (P (U = u)) P (U = u). u. "$#P"&%.
(23)
(24)
(25) 0. GIHKM&Y\FRbht[]TU[d\aRbWa MR:TXVU\Fq: | V ) = H(U, V ) − H(V ) :bWfR:TX[bM6gTXHKMW¢FMV WfeaMRiKL|rMDV \a|K[]TXgVUM :K[]VXMDh^TX\^hKMDgXYVX[]|+M UH(U c:HKMDR>TUHbM-WfdKM\f [dgWfdVUM6Waht^iRK\-R $K\FV[dRbgsTXWfRrYMaci[] [kgW hKMTUMDVUL[dRK[kgnTX[dYKRbYTU[d\aR&\f V ciTUHKMDR&TUHK[kgY\FRbht[]TU[d\aRbWa+VMDR:TUVX\aqi[dgIRiKdBcKWFgRK\L\FVUM@[]RK\aVXLWfTUU[d\aR[dg VXM :K[dVXMDhTU\2hKMDgXYVX[]|+M U HKMR V [dg iRK\-RzRTXHKMY\aR:TUV WfVXaca[²TUHbMWfVXMo[]RrhtMq+MRbhKMR:TDc:iRK\-[]Rbe ht\iM6g1RK\fTTXMda\F¬WfRi:TUHb[]RKe¥Wa|r\FtT U WaRbhªTUHKM>Y\aRbht[]TU[d\aRrWf/MR:TXVU\Fq:¥[kgWfdL\FgsTM :bWfTX\TUHKM V MDR:TUVX\aqi`[T gsMDn zKVIM6W-TXKVUMgUMdMDYTU[d\aR&[dg|rWagUMDh\aRTUHKMY\aRbht[]TU[d\aRrWfL2KTUbWa[]RK\aVXLWfTU[d\aR I(U, V | W ) = H(U | W ) − H(U | W, V ). GIHK[kg-WfdKM :bWaR:TU[]bMDgHK\- L2rY H>[]RK\aVXLWfTU[d\aR>[kgIgsHrWfVXMDh|+MTnMMR WaRbh cieF[]FMR`TXHKM-Wa]KM \a W >«RK\fTXHKMV2W¢TX\¦gsMDM^[]TDcWag1[]T[]T2htMDY\aLqr\:gsM6h¬Wa|r\-FMac[kgWFg@TUHKUM>ht[-²MVXVMRbYM`|rMTnMDMRTUHKM [dRt\FVUL`W-TX[]\FRVXM:K[dVXMDhTU\`htMDgXYVX[d|rM U ea[daMDR W cbWaRbhTUHKM[dRt\aVXL`W-TX[]\FRTX\>htMDgXYVX[]|+M U ea[daMR&|+\fTUH WfRbh P WfRbh YWaVUVX^TUHKM1gUWaLM[dRt\aVXL`W-TX[]\FRWf|+\atT c:TUHbMTn\TXMVXL`g\FR>TXHKM1VU[deaH:TWaVUM W M :bWf=cWfVRbhªTUHbVMY\aRbhK[WTX[]\FRbWf/LtTUbWa/[]RK\aVXLWfTU[d\aR¬[dg1§DMVX\bzURªTUHKM>Y\aR:TUV WfVX&[I|r\aTUH V WfRbh W |bVU[dRKe[]RK\aVXLWfTU[d\aRcKWaRbh[!TUHK\:gsM1[dRt\aVXL`W-TX[]\FR&WaVUMY\FLqK]MDLMR:TXWaVUFcFTXHKMht[ -+MDVUMDRbYM1[kgkWfVXeaMa BA@. FLK
(26) NI GJI)FLK H 'G'H . K F .
(27). HGJI)F K H'I I HQGJIDFLK. . GIHbMoL`Wf[dR`ea\FWaK\fM6W-TXKVUMgsMD]M6YZTU[d\aR`[kg/TX\2gUMdMDYZTWgsL`Wa]+gsK|rgsMT \fM6W-TUbVUM6gTUHbWfTYDWfVXVU[dMDgWFg/L2rY H d[ Rt\FVUL`W-TX[]\FR¦Wagqr\:gUgU[]|b]MFGIHKM2bTX[]L`W-TXM2eF\FWf \FKkh|rMTU\>L[dRK[dL^[d§M Hˆ Y | F , . . . , F ¡tT@TUHb[dg1MiqbVUM6gUgU[]\FR¦YWaRªRK\aT@|+MM6gnTX[]L`WfTUMDh¬[]TUH¬WTUV Wf[dRK[dRKegUMT1\fVUM6Wfd[dgsTU[kYgU[d§MWFg@[T1VUM :K[]VXMDg TXHKM¦M6gnTX[]L`W-TX[]\FR \f 2 qKVU\F|bWf|b[]d[TX[]M6g $KbVsTXHKMVXL\aVXMacIMFMR [TXHKMVXM¥MVXM¥W¢tgTX\(HbW¢aM¥WaR M6gnTX[]L`WfTU[d\aRct[]TXgL[dRK[dL^[d§DWfTU[d\aR&\FKkh|+MY\FLqKtTXWfTU[d\aRbWa]d`KR:TUV WaYTXWa|K]MF «T TUHKM\fTUHbMV MiTUVXMLMac¢\aRbM Y\FKdh2ht\WTXVU[di[dWaaV WfRbhK\aL gXWfLqKd[dRKeoHK[kY H\aKkh1MRbgUKVXM/TU\@gs\FLM MiTUMR:To[dRbhtMDqrMDRbhtMRrYM|+MTnMMDRM6W-TXKVUM ¨ [ht[ -²MVXMR:TTniqrM6g\fMDWfTUKVXMDgWaVUM1M FrWfd]VUMDqKVUM6gsMDR:TUMDhr© |btT\aKkhRK\aTW-T1WfdTXWaaM2YDWfVXM\f/qKVXMDht[kYZTX[]FMq+\-MDVDoG \htM6Wf []TUH¦TUHbWfTDc+\aRKM2Y\aKkh¥gUMdMDYZTTUHKM |+MDgsT^MDW-TXKVXMDg^WaYY\aV ht[dRKeTU\ gs\FLM>MDgsTU[dL`W-TUM&\fTXHKM[dV[]Rrht[]i[khtbWaqKVXMDhK[dYTU[daMq+\-MV6¬GIHKM&L`Wf[dR MDWaiRKMDgXg@\f TXHK[kgWfqKqbVU\:WaY H¦[kg@TUHbWfT2Wf]TUHK\FKeaH[]TTXWfFMDg1YWaVUM^\fI[dRbht[d:[khtbWa/qKVXMDht[kYZTX[]FM^q+\-MV6c[T hK\:M6gRK\aT^W¢F\a[kh W-T^Wa]VUM6htKRbhbWfRbYWaL^\FRKe¥TXHKM&gUMdMDYTUM6hMDW-TXKVXMDgD¬¡IWagU[dYDWfd]Fc \FRKM\abdh qK[kY L`WaR:>gU[dL^[dkWfVIMDWfTUKVXMDgDcKWagTXHKM\aRKM6gYWaVUVXi[]RKe^W^]\aTI\a [dRt\aVXL`W-TX[]\FR&WaVUM@][daMD]>TU\|rM\a!WYMDVsT Wf[dR Tniq+Ma3 $K\FVWWaYM@htMTXMDYTU[d\aR`T WagU\FV [dRbgsTXWfRrYMaciM6hteaM6g\aR>TUHKMMaMD|KVX\-ogWfRbh`TUHKM@L\atTXH>\FKkh`|rM TXHKM\aRb]>\FRKMDgY\aLq+MTU[]TU[daMF M¥qKVX\aq+\FgUMWfR [dRFTXMVXLMDht[kW-TXM&gU\adtTU[d\aR zKV`WfqbqKVU\:WaY H hKMDWfkg^[TXH TUHKM&TUV WahKM
(28) \ - |+MTnMMDR [dRbht[di[dhKbWf/q+\-MDVWfRbh¬[dRbhtMq+MRrhtMRbYM`|iªY\FL^qrWfVX[]RKe&MDWFY H¬RbMMDW-TXKVXM^[]TUHTXHKM`\aRKM6g1WfdVUM6Waht qb[dY FMDh¬MgXW¢¥TXHbW-TW&M6W-TXKVUM [dgea\i\th¬\FRKdª[] ˆ F | F ) [kgHK[deaH \FVMFMVX F Wa]VXMDWFht qb[dY FMDh GIHb[dgL^M6WfRbg`TUHbWfT F [kg`F ea\i\th \aRKd []@[]TYDWfI(Y, VXVU[dMDg[dRt\aVXL`W-TX[]\FR Wa|r\FtT coWaRbh [1TUHb[dg [dRt\FVUL`W-TX[]\FRHbWag RK\fT |+MMDRYWaKeaH:T|iWfRi2\a+TXHKM F Wa]VXMDWFht2qK[kY aM6h¤¥\aVXMI\aVXL`WfdY]FcaMoqKVX\aq+\FgUM TXHKM1\add\-[]RKe^[]TUMV W-TX[]FMgUY HKMDLM n(1). M +1. ?. ?. ?. ˆ n(1) = arg max I(Y, Fi ) i. PQPSR#TJU VU4W. n(M ).
(29) . n(k + 1) = arg max i. . ˆ min I(Y, Fi | Fn(l) ) l≤k. . ¡¬TXWfi[dRKeTUHbM>L[]RK[dL2bL \aTXHKMY\aRbht[]TU[d\aRrWfLtTUbWa [dRt\aVXL`W-TX[]\FR\FR(Wfd/MDWfTUKVXMDg2WfdVUM6Waht qb[dY FMDhcMMDRbgUKVUMTUHrW-TWRbMuM6W-TUbVUM^[dg1|+\fTXH¬[dRt\FVUL`W-TX[]FMWaRbh¬hK[ -²MVXMR:T@TUHbWaRªTXHKMqKVUM6YM6ht[]Rbe \FRKMDgD GIHKM¥Y\aLqKKTXW-TX[]\FR \aTXHK\FgUM¥gUY\aVXMDgYWaR |+M¦ht\aRKM¥WFYYbVXWfTUMd(WagTXHKM VUM Fb[]VXM\FRKd TUHKM M6gnTX[]L`WfTU[d\aR&\f/ht[dgsTUVX[d|KtTU[d\aRrgI\f TXVU[dqKdMT g\f|r\i\F]M6WfR-WaVU[kWf|KdMDgD BG'H. BA. . K
(30) F HQGJIDFLK. H BI I . HGJI)FLK. G \¥Y\FL^qrWfVXM\FKVWfqKqKVX\FWFY H¦TX\gUMdMDYTU[d\aR|bWagUMDhª\aR[]Rrht[]i[khtbWaqKVXMDht[kYZTX[]\FR¬q+\-MDVDcMHbW¢aMWfkgU\ [dLqKdMLMR:TUM6h W¥LMTUHK\th(Hb[dY H(qb[dY tg2TUHKM K MDWfTUKVXMDgL`W-t[dL^[d§[dRKe¬[]RbhK[]i[khtbWfdd¬TXHKM[dV^L2tTXbWf [dRt\FVUL`W-TX[]\FR[TXH`TUHKM@YkWagXgTU\qKVXMDht[kYZT Iˆ Y, F PSR`TUHbMoVUM6gsbTgUMDYTU[d\aRbgDc:MYDWfdKTXHK[dg LMTUHb\ih ¤¦Pn¤ \FVo¤¥tTXbWfPSRt\FVUL`WfTU[d\aR¤¦W-t[dL^[d§DWfTU[d\aR n(l). BA. F. . B I)G . GIHbMIY\aLqKtT W-TX[]\FRbWfKY\FgsTXg\f£¤¥Pn¤ WfRbh^«hKWF¡\i\:gnT/WfVXMIM :K[d¢Wa]MDR:THbMRTUHKMogUMT\a+WfdKW¢-Wf[ddWa|KdM M6W-TXKVUM6g2[kgbRK[]TUMWfRbh(VUMDLWa[]R [dhtMDR:TU[kYWfWfTWa][TXMV W-TX[]\FRbgMgsTU[ddIhKMRK\aTUMHbMVXM N TXHKMTU\fT Wf RiKL|rMDV`\fMDW-TXKVXMDgDc M TXHKM¥RiKL2|+MV`\aM6W-TXKVUM6g^MIWfR:TTU\ gsMD]M6YZT6cWaRbh T TUHbMgU[d§M\f@TUHKM TXVXWa[]Rb[]RKe`gUMT6 ¡\fTXH|r\i\FgsTU[dRKe>WaRbh¦£¤¥Pn¤ HbW¢aMWL`Wf[dR]\i\aq&VXMq+MDW-TXMDh TU[dL^M6gcbHK[dY HqK[kY tgIMDWfTUKVXM1\aRKM Wf TUMDV^WaRK\fTXHKMV6 $b\aV2M6WaY H([]TUMDVXWfTU[d\aR(\aTUHK[kg]\i\aq c«hKWa¡\i\FgsT2MVXM :K[dVUM6gTU\Y\FL^qbtTUMW¦M[deaH:TUM6h MDVUVX\aV&V W-TUM¬\aV¥M6WaY H \aRKM\f^TUHbM N W¢-Wf[ddWa|KdMªM6W-TUbVUM6gc@HK[kY HY\FgsTXg O(N × T ) \aq+MV W-TU[d\aRrg 1 MDWaVURK[dRKe^[TXHª£¤¥Pn¤ VUM Fb[]VXMDgTX\Y\aLqKtTXM1TUHKM ˆ ct\aVMDaMDVU HKMDVUM [k kgTUHbM¦[]RrhtM \f@TUHKM¦kWagsTgUMdMDYZTXMDh MDWfTUKVXMa GIHHK[dg>Y[dg| FTXVUbM¥,WFg`F d\aRKe Wag>M¦YD1Wf≤R ai MD≤Mq NWTXWf|b]M [TXHTUHKMYKVXVUMDR:TIL[dRK[dL2KL -WfdKM\f!TUHKMY\aRrht[TX[]\FRbWf L2tTXbWf[dRt_ ˆ s[i] = min \FVUL`WfTU[d\aR \aV>I(Y, MDaMDVUF | FM6Wf()M6W-TXKVUMª\f1TUHKM¬Y\aLqKdMTXMªgUMTD (irY H W(Y\FL^qbtTXWfTU[d\aR VUM :K[]VXM¦WfkgU\ \FqrMDVXWfTU[d\aRbgD $![dRbWfddacK|+\fTXH&LMTUHK\thKgoY\FgsT[dg O(M × N × T ) O(N × T ) ¬ I.0 n(k). l≤k. . . i. i. n(l). . ADC. . HQGH M GM. MoHbW¢FMIbgsM6hTUV Wf[dRK[]RbeWfRbhTUM6gnTgsMTXg/M:TXVXWFYZTXMDhVU\FL Tn\1dWaVUeFMIgUMT g/\fqK[kYZTXKVXMDgD!GIHK\:gsM\FVU[dea[dRbWa |b[]e`gUMTXgIMVXM1WagXgsMDL2|KdMDh|iY\F]dMDYTU[dRKe^WMD TXHK\abgXWfRrhKggUYMRKM6gVU\FL TUHKMM|WfRrhL`WfVXi[]Rbe|i HrWfRbh¬TUHbM`]\tYWfTU[d\aRbg1\aMDaMDgWaRbhL\atTXH\FR MaMDVU¥i[kgs[d|KdMWaYMa $KVX\aL MFMVXWaYM`MeaMDRKMV W-TXM TXMRgUL`WfdbeFVXW¢tgXYWfdMWaYM[]L`WfeFMDg |:^V WfRbht\FL[]§D[]RKe[]TXg qr\:gsMF/GIHK[dg ]M6Wah^TX\TUMDRbg\aTUHK\FbgXWfRbhKg/WaYM qb[dYTUKVXMDgD¬M`HbW¢FM^WadgU\Y\addMDYZTXMDhY\FL^qb]M¬gUYMRKM6g ¨ \FVUM6gnT gc|bK[]kht[dRKeFgDcbVURK[]TUbVUM6gcMTXYa © VX\aL Hb[dY HMoHbW¢aMWatTU\FLWfTU[kYWa]dMiTXVXWFYZTUM6hTUMRrg\f²TXHK\abgXWfRrhKg/\f²|bWFY :eFVU\FKRbh ¨ RK\FRt_=WaYM¢©!qK[kYZTUbVUM6g «] TUHK\:gsM`qb[dYTUKVXMDgWaVUM\agU[d§M qK[]iMDdgDc WaRbh :bWfR:TX[bM6h[]R eFVXW¢tgXYWfdM^]MDaMkgD $bWFYM6g HrW¢aM@|rMDMRVUMDea[kgnTXMVXMDhVU\FKeaHKdacKgU28\^TU×HbWf28TTUHKMYMR:TUMDV\f!TUHKMMDaM6g[kg[dR¥256W 2 × 2 YMR:TXVXWag :bWaVUMFciTUHKM "$#P"&%.
(31)
(32)
(33) . x. $![deaKVXM¥y GIHKM`Tn\¥KqKq+MV2VX\-oggUHK\-MKWfLqKdMDg\aI|rWaY ieaVX\aKRrh¦qK[kYZTXKVUM6gcWfRrhTUHKM`Tn\¥d\-MDV XV \-oggUHK\-MKWaL^qb]M6g\a/WaYM^qK[kYZTXKVUM6g1«od!TUHK\:gsM[]L`WaeaMDgWfVXM2eFVXW¢tgXYWfdM1\fgU[d§M 28 × 28 qK[tMDdgDc MiTUV WaYZTXMDh VX\aL Y\aLqKdMTXM¬qK[kYZTXKVXMDgTXWfFMR \aR TXHKM u'$bWaYMDg&WfVXMªVX\aKeFHKd YMR:TUMDVUM6h WfRrh gsTXWaRbhKWaVXht[d§M6h>[dR¥gs[d§MF hK[dgsTXWaRbYM|+MTnMMR¬TUHbM`MaM6g1[dg 10 TX\ 12 qK[tMDdgWaRbh¦TXHKMTU[d]T[dg1|+MTnMMDR −10 WaRbh +10 htMDeaVXMMDg ¨ YZnbeFKVXM^y¢©Z $b\aVoM6WaY HMtq+MVX[]LMR:T|+\fTXHTUHKMTUV Wf[dRK[]Rbe>WfRbh&TXHKMTUM6gnT@gsMTXgY\aR:TXWa[]R []L`WfeFMDgDcKVX\aKeFHK]&Wag 500 L`WaR:\f Hb[dY H¥WfVXMWaYM6gWfRbh&RK\FRt_=WaYM6g . A@. . A.
(34) .
(35). HG M. M rgsM MDW-TXKVXMDg!gs[dL[]kWfV TU\oTXHKMMDhteFM htMTXMDYTU\aV g[dR $Q 1~byfc4$Q 1~F}4"=GIHKMWaVUMM6WagUTX\Y\aLqKtTXMWfRrh VX\a|bbgnTTX\`[]ddKL[]RbWfTU[d\aR¥-WfVX[kW-TU[d\aRrg WaY H&MDWfTUKVXM[kgW|+\:\F]M6WfR&KRbYTU[d\aR¥[dRbhtMiM6h|iW`d\tYW-TX[]\FR []R¦TUHKM VXMMVXMRrYMV WfLM2\aTUHbM2[dLWaeaMFc²Wht[dVUM6YZTU[d\aR HK[kY HªYWaR¥TXWaaM ht[-+MDVUMDR:T (x, -Wa]Ky)M6g ¨ YZn!bea28bVUM6×g28}WaRbh`NF© WfRbh`W1TX\adMV WfRbYM t HK[kY H`[dgWfR`[dR:TUMeFMVd-Wa]KM|rMTnMDMR 1 Wa8Rbh 7 /GIHKM TX\adMV WfRbYMY\aVXVXMDgUqr\FRbhKgTU\`TXHKM2gU[]§DM\f!TUHKM2RKM[deaHi|+\aVXHK\i\ihHKMDVUM1TXHKMMDhKeaMHbWFgITU\`|+MqKVXMDgUMR:TI\aV TXHKMMDW-TXKVXMTX\|+MM :bWfrTU\ 1 ¨ YZnbeFKVXM@}a© / MRrYMaciMHrW¢aMWgsMT\a 28 × 28 × 8 × 7 = 43, 904 M6W-TXKVUM6g J H3IK IDK H3K
(36) SG M GJIDK. KVX[]Rbe1TUHKMTXVXWa[]Rb[]RKe1\a+TXHKMq+MV YMqKTUVX\aRc:Y\iM .>Y[dMR:T gWfVXMIRK\fT |+\abRbhWfRbh^TXHKMRiKL2|+MV\a²TUV Wf[dRK[]Rbe d\i\aqbg2[dg2][dL[TXMDh(TU\ 100 [TXMV W-TX[]\FRbg[dR TUHbMRK\FRt_ gUMqrWfV Wf|KdM`YWagUMa¦GIHbMY\iM.`Y[]MDR:TXg\aTUHKMRbWf[daM ¡IW¢FMDgU[dWaRWfVXMY\aLqKKTUMDhht[dVXMDYZTX]WaghtM6gUYVU[d|rM6h[dR¦}t }t VXVX\aV V W-TXMDgWfVXMW¢aMDVXWaeaM6h2\FR VX\aKRrhKg\FVM6WaY H`Miq+MVX[dL^MDR:TD3$K\FV MDWFY H>VX\aKRbhcaTXHKMTXVXWa[]Rb[]RKe gUMToWaRbhTUHKM1TXMDgsTgsMTWaVUM1M:TXVXWFYZTXMD25hVXWaRbht\aLdVU\FL TUHbM\aVX[]eF[]RbWa²dWaVUeFM@hKWfTXWgUMTXg ¨ YZn 5Xwbdy6©. PQPSR#TJU VU4W.
(37) y6~. . x. x. y. y. t. ![deaKVXM} GIHbMM6hteaMIhtMTUMDYTU\FVXg!MIWaVUMbgU[]Rbe@WaVUMIYVXbhtMI|KtT/[dRi¢WaVU[kWfR:T!TU\Y HbWaRKeaM6g [dR^[]d]bL^[dRbWfTU[d\aR GIHbM¥qK[dYTUKVXM¥\aR TXHKM¦dM TgUHK\-og^TUHbMªYVX[TXMVX[]\FR \FVWHK\aVX[d§\aR:T WfMDhKeaM¥d\iYDW-TXMDh []R y) GIHKM hKMTUM6YZTX\aV/VXMDgUqr\FRbhKg[]rTXHKMgU[hK[ -²MVXMRbYMDg/|+MTnMMDRqb[tMkgY\aRKRKM6YZTXMDh^|i2WTUHK[dR>gsMDeaLM(x, R:TWfVXMI]M6gUgUMV [dRWa|bgU\adtTUM-WfdKM TXHbWfR2TUHKMIht[ -+MDVUMDRbYM|+MTnMMR2TUHKMIqK[]tMkgY\aRbRKMDYTUMDh|:1TXHKMTUHK[kY gUMeaLMDRFT6 GIHKM VXMkW-TX[]FM-WfdKMDg\fTUHbMTn\>qK[]iMDdg@Y\FRKRKM6YZTUM6h|iTXHKMTXHK[kY d[]RbMhtMbRKMTXHKM2q+\akWfVX[]Tn\a!TXHKM2M6hteaM ¨ hKWfVX^TU\][deaH:TI\aV d[deaH:TTX\hKWaVUK©GIHbMqK[kYZTXKVXM\aRTUHKM@VX[]eFHFTIgsHb\-og/TUHbMRKMD[]eFHi|r\FVUHK\i\thHKMDVUMTUHKM M6hteaM1HbWFgTU\|+MhtMTXMDYZTXMDh\aVoWM6W-TUbVUM\a TU\F]MDVXWaRbYM t = 5 TU\`VXMDgUqr\FRbh $. $![deaKVXM2N GIHKM\aVX[]eF[]RrWfeFVXW¢tgXYWfdM@qK[dYTUKVXMDg@WfVXMgUHK\-R¥\aRTXHKM2dM T6GIHKMMD[]eFH:T|K[dRbWfVXL`Waqbgo\FR XT HKM2VX[deaH:TgUHK\- TXHKM^VUM6gsq+\aRbgUMDgo\fTUHKM^MDhKeaM2htMTUM6YZTU\FVXgW-TMDaMDVU]\tYWfTU[d\aRbg[]R¦TUHKM 28 × 28 VXWaL^MFc \FVMFMVX`\aRKM1\f!TUHbM 8 q+\FgXgU[]|KdMht[dVUM6YZTU[d\aRrgWfRbhqr\FdWaVU[]TU[dMDgD. "$#P"&%.
(38) yFy.
(39)
(40) . . £kWagXgs[]bMV G V Wf[dRK[]Rbe^MDVUVX\aV G MDgsToMVXVX\aV «hKWF¡\i\:gnT 0% 1.45% £¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR ¡W¢FMDgU[dWaR 0.52% 1.52% £¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FRuqrMDVXYMqtTXVU\FR 0% 2.28% «hKWF¡\i\:gnTMDWfTUKVXMgsMD]M6YZTU[d\aR qrMDVXYMqtTXVU\FR 0% 2.46% «hKWF¡\i\:gnTMDWfTUKVXMgsMD]M6YZTU[d\aR $¡W¢FMDgU[dWaR 0.4% 3.51% ¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR uqrMDVXYMqtTXVU\FR 3.56% 8.28% ¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR ¡W¢FMDgU[dWaR 5.58% 8.54% G!Wf|b]My VXVX\aVV W-TUM6g[TXH 50 M6W-TXKVUM6gI\aRTUHKMWFYYKVXWfTUMTUV Wf[dRK[dRKe`gsMTD Z . BADC. . I. . BA@. . I. . K GM:I GH JHG G JH3IDKNIDK M GJM. GIHbMbV gnT@VU\FKRbh¥\aMtq+MVX[]LMR:T gIWaghtMDgU[]eFRKMDh¥TU\Y\aLqbWaVUMTXHKM^qrMDVs\FVUL`WfRrYMDg\f\FKVLMTUHK\thKg []TUHTn\\fTUHbMV1MDWfTUKVXM>gUMdMDYZTX[]\FR¬LMTUHK\thKg ¨ «hKWa¡\i\FgsTbgUMDh WagW&M6W-TUbVUM`gUMdMDYZTX\aV6c WfRrh¤¦Pn¤ Hb[dY H [kgW gU[]LqKdM gsMD]M6YZTU[d\aR |bWagUMDh \FR [dRbht[d:[khtbWaLtTUrWf1[]RK\aVXLWfTU[d\aR YnS5XNK NF©>HKMR TUHKM M6W-TXKVUM6g WaVUMY\aL|K[]RbMDh[]TUHWq+MV YMDqtTUVX\aR !GIHKMVUM6gsbT g ¨ YnbeabVUMowWfRbhT Wf|KdMy¢©gUHK\- TUHbWfTWag W>MDWfTUKVXM^gUMdMDYTU\FVDc²«hKWa¡\i\FgsT@hK\:M6g@RK\fT1qrMDVs\FVUL |+MTUTUMV@TXHbWfR¬\aKV@LMTUHb\ih2¡tT@[] MrgsMTUHKM M[deaH:T gY\aLqKtTXMDh&|i«hKWF¡\i\:gnT[]TXgUM]nctTUHKMeF]\F|bWfYdWFgUgU[rMVI\atTXqrMDVs\FVUL`gI\FKVXgD PSR&TUHbM1gUMDY\aRbhVX\aKRbhMY\aL2|b[]RKM@TXHKM@MDW-TXKVXMDgI[]TUHWRrWf[daM1¡W¢FMDgU[dWaRYkWagXgU[bMDVDci[dRbgsTUMDWFh\a Wq+MV YMqKTUVX\aR2GIHb[dg1Miq+MVX[dL^MDR:T1gUHK\-ogTUHbWfTM6W-TXKVUM6g1gsMD]M6YZTUM6hª|iTXHKM£¤¥Pn¤ dMDWFh¥TX\WRbWa[]FM ¡IW¢FMDgU[dWaRWagIq+\-MDVsbWagTUHbM«hKWa¡\i\FgsTIYdWFgUgU[]bMV ¨ Yn/beFKVUM ;^WfRrh>T Wf|KdM`y6© GIHK\:gsMVUM6gsbT gWfVXM2Y\aRbgU[dgsTUMDR:TDGIHKMqrMDVXYMqtTXVU\FR¦gsJ -+MDVXgVU\FL []TXg@gU[]LqKd[dY[Tn oWFggU\:\FR¦WagTUHKM RiKdbMDVUVX\aV/V W-TXM[dg \a|tT Wf[dRKMDh\FRTUHKMoTUV Wf[dRK[]RbegsMTDc:[TgsTU\FqbgDcFdMDWFht[]Rbe1TU\\aV gnTMDVUVX\aV/V W-TXMITUHbWaR`HbW-T L`Wfi[dL2bL LWaVUeF[]R WfqKqKVX\FWFY HKMDg^Y\FKkh ht\r «hKWa¡\i\FgsTea\iMDg\aR LWft[]L[d§[dRKeªTXHKMVUM6gsq+\aRrgsM6g2\FR TXVXWa[]Rb[]RKegXWfLqKdMMFMRHKMDRTUHKMoRiKdbMDVUVX\aVV W-TXM[dg VUM6WaY HKM6h2\FRTUHKMTXVXWa[]Rb[]RKegUMT6GIHb[dg LWft[]L[d§MDg TXHKML`WaVUeF[]R |+MTnMMDR TUHKMYkWagXgU[rYDW-TX[]\FR|+\abRbhKWfVX WfRbh TUHKMTUV Wf[dRK[]RbeªgUWaLqK]M6gWaRbh eFMRKMDVXWa][d§M6g |+MTUTUMDVD/GIHKM@VUM6gsbT[TXH>TUHbM@«hKWF¡\:\:gnT RbWf[daM@¡W¢FMDgU[dWaR`WfVXMY\FRbgs[kgsTUMR:TTU\i\bctY\FRbgU[dhtMDVU[dRKeTXHbW-T «hbWa¡\i\FgsTIht\iM6gIRK\fTT WfaMYDWfVXM\a TXHKM[]RrhtMq+MRbhKMRbYM|rMTnMDMRMDW-TXKVXMDgD GIHKMªaMVX(|bWFh qrMDVs\FVUL`WaRbYM\f1TXHKMª¤¥Pn¤{YVX[]TUMVX[d\aR YWaR |+M¦MiqbdWa[]RKM6h |: d\:\Fi[]RKe WfT>TUHKM Y Hb\FgUMR¥M6W-TUbVUM6g1«g@Mtq+MDYTUMDh¦TUHKMD¦WfVXM2HK[deaHb]¥gU[]L[ddWaV|+\fTXHª[dRªd\iYDW-TX[]\FRbgWaRbhªhK[]VXMDYTU[d\aRc+TUHbM WaVUM@|bWFgs[kYWa]dWfdhtMTUMDYTU[dRKeTUHbMKqKq+MVqbWaVsT\f!TUHbMgsiKdB K G I GSK'FLIM G H IK IK. M GJM. G \TUMDgsTTUHbM@VX\a|bbgnTXRKMDgXg\f TUHKMY\aL2|b[]RbWfTU[d\aR\f/£¤¦Pn¤ WaRbh>TXHKMRbWa[]FM@¡IW¢aM6gs[kWfR cFMht[dh&WTUHK[dV h VX\abRbh&\f/Mtq+MVX[]LMR:T go[]TUH¦RK\F[dgU>TXVXWa[]Rb[]RKehKW-T WKcbiRK\-R&TX\>|rM^ht[.`YK]To\aV|r\i\FgsTU[dRKegXY HKMDL^M6g MeaMDRKMV W-TXMDhTXHK[dgTXVXWa[]RK[dRKegsMT@|i b[dqKqK[dRKe¥W-T1V WfRbht\FL ; \aTUHKM^TXVXWa[]RK[dRKedWa|rMDdgDP TYVXMDWfTUMDg W ht[ .`YK]TgU[TXbW-TX[]\FR \FVdMDWaVURK[dRKe(LMTUHK\thKg&HK[kY H TXWaaMYWfVXM¦\a2\atTX][dMV gDcWagTUHKMDVUM¬[dg ;
(41) \a TXHKMLcIht[kgnTXVU[d|KtTXMDh KRK[]\aVXL^d WaL^\FRKe¦TXHKMTUV Wf[dRK[dRKeqr\FqKKkW-TU[d\aR pM6gsK]TXg`WaVUMgUKLLWaVU[d§M6h(\FR reaKVXMDg WfRbh TXWa|KdM¥}K ¡\aTUH £¤¥Pn ¤ q+MV YMqKTUVX\aR WaRbh «hKWa¡\i\FgsT^q+MVU\aVXL FMVX |bWahtdacIWag PQPSR#TJU VU4W.
(42) y¢}. . Train. Test. 0.2. 0.2 CMIM feature selection + perceptron MIM feature selection + perceptron AdaBoost feature selection + perceptron AdaBoost. CMIM feature selection + perceptron MIM feature selection + perceptron AdaBoost feature selection + perceptron AdaBoost 0.15 Error rates. Error rates. 0.15. 0.1. 0.05. 0.1. 0.05. 0. 0 10. 20. 30 40 Number of features. 50. 60. 10. 20. 30 40 Number of features. 50. 60. $![deaKVXM/w GIHKMq+MVU\aVXLWaRbYM6g\fiTXHKMTn\ITniqrM6g\ftM6W-TXKVUMgsMD]M6YZTU[d\aRrgY\FL2|K[dRKMDh1[]TUHWqrMDVXYMqtTXVU\FR TXVXWa[]Rb[]RKerGIHKMYbVUFMDgIgsHb\- TUHbMMVXVU\FVIVXWfTUMDgItgDTUHKMRiKL|rMDVo\f/Y\FRbgU[dhtMDVUM6h>M6W-TXKVUM6gGIHbMMVXVU\FV \FR TUHKMTXVXWa[]RK[dRKe(gUMT>[kg>gUHK\-R \FR TUHKM¦dM T&WfRbh TUHbM¥MVXVU\FVXg^\aR TXHKM¥TUM6gnTgUMT>\FR TUHKM¦VX[deaH:TD « q+MV YMDqtTUVX\aR |bK[]]T[]TUH TUHKMMDWfTUKVXMDggsMD]M6YZTUM6h |i £¤¦Pn¤ ¨ TXHK[dY Y\aR:TU[dRiK\abg2][dRKM¢©q+MVU\aVXL`g2Wag MdWagWqrMDVXYMqtTXVU\FR¦|bWFgsM6h¥\aR¥TXHKMM6W-TXKVUM6gY Hb\FgUMR¥|i¥«hKWa¡\i\FgsT ¨ TXHK[dRhKWFgsH¥d[dRKM6© (iTU[d]=c²TUHKM qbKVUM«hKWF¡\:\:gnTIVXMDWaY HbMDgIW^|rMTsTUMDVoMVXVX\aVIVXWfTUM ¨ TXHK[kY hKWFgsH&][dRKM¢©Z Train. Test. 0.2. 0.2 CMIM feature selection + bayesian MIM feature selection + bayesian AdaBoost feature selection + bayesian AdaBoost. CMIM feature selection + bayesian MIM feature selection + bayesian AdaBoost feature selection + bayesian AdaBoost 0.15 Error rates. Error rates. 0.15. 0.1. 0.05. 0.1. 0.05. 0. 0 10. 20. 30 40 Number of features. 50. 60. 10. 20. 30 40 Number of features. 50. ![deaKVXM; GIHKM>q+MVU\aVXLWaRbYM6g1\fTUHKM>Tn\Tniq+MDg\aM6W-TXKVUM>gUMdMDYTU[d\aRbg2Y\aL|K[dRKMDh[]TUH(WRbWa[]FM ¡IW¢FMDgU[dWaR&YdWFgUgU[rMV6GIHKM2YKVXaMDggsHK\- TUHbMMVXVU\FVVXWfTUM6gtgTUHbMRiKL2|+MV\fY\FRbgs[khtMDVUM6hM6W-TXKVUM6g GIHbMMDVUVX\aV^\aR TUHKM&TUV Wf[dRK[dRKe gsMT[kggsHb\-R \aR TUHKMdM TWaRbh TXHKMMVXVU\FVXg2\aR TUHKM&TUM6gnT>gUMT\aR TUHKM VX[deaH:TDGIHKMRbWf[daM¡IW¢aM6gs[kWfR&|bWFgsM6h\aRTUHbMMDWfTUKVXMDggsMD]M6YZTXMDh|i¥£¤¦Pn¤ ¨ TXHK[dY Y\FR:TU[dR:b\abgd[dRKM6© q+MVU\aVXL`gWagMD]bWFg/qKKVXM«hKWF¡\:\:gnT ¨ TUHK[kY ^hbWagUH`][dRKM6©c:WaRbh^|rMTsTXMVTUHbWaRWRbWf[daMo¡IW¢aM6gs[kWfRbgU[]RKe TXHKM1MDWfTUKVXMDgY Hb\FgUMR|i&«hKWa¡\i\FgsT ¨ TUHK[dR¥hKWagUH][dRKM¢©Z $. "$#P"&%. 60.
(43) y6N.
(44)
(45) . Train. Test. 0.2. 0.2 CMIM features selection + Bayesian CMIM feature selection + perceptron AdaBoost. CMIM features selection + Bayesian CMIM feature selection + perceptron AdaBoost 0.15 Error rates. Error rates. 0.15. 0.1. 0.05. 0.1. 0.05. 0. 0 10. 20. 30 40 Number of features. 50. 60. 10. 20. 30 40 Number of features. 50. $![deaKVXM £\aLqbWfVX[kgs\FR`\f q+MVU\aVXL`WfRbYMDg[TXH&W2RK\a[kgsTXVXWa[]Rb[]RKe^gsMTD/GIHKM1YKVXaM6ggUHK\- TXHKM1MVXVU\FV V W-TXMDg/tgD!TUHKMR:bL2|+MV \a Y\FRbgs[khtMDVUM6h2M6W-TXKVUM6g/\FVW1TXVXWa[]RK[dRKe2gUMTHK\FgUMkWf|+Mkg HbW¢FM|rMDMR r[]qKq+MDh []TUH(W¦qKVU\F|bWf|b[]d[Tn ¦GIHKM>MDVUVX\aV\FR TUHKMTUV Wf[dRK[]Rbe¦gsMT[kg2gUHK\-R \aR TUHbMdM T^WaRbhTUHbM>MVXVU\FVXg \FRTUHKMTUMDgsT@gUMT\aRTU5% HbM2VX[]eFHFT6«goMtqrM6YZTUM6hc²«hKWa¡\i\FgsT ¨ hKWFgsH¥d[dRKM6©o\-aMVUKT gWfRbhVXMDWFY HKMDgoW>R:b] MDVUVX\aVVXWfTUM\aR¦TXHKM^TUV Wf[dRK[dRKegUMTWfRbh¬WHb[]eFHªMVXVX\aVV W-TXM\FR¥TUHbMTXMDgsT1gUMT62GIHKM^qrMDVXYMqtTXVU\FRª|bWFgsM6h \FRTUHKM£¤¥Pn¤ M6W-TUbVUM6g ¨ Y\aR:TU[dRiK\arg/TUHb[]R>d[dRKM6© ht\iMDg|bWah|+\fTXH>\aR`TXVXWa[]Rb[]RKeWfRbh^TXMDgsTU[dRKebc:|KtTTUHKM gXWfLMMDW-TXKVXMDg Y\aL|K[]RbMDh`[]TUH&WRbWa[]FM¡IW¢aMDgU[kWfR ¨ Y\aR:TU[dRiK\arg/TUHb[dY d[]RKM¢© hKML\aRbgsTUV W-TXMW2VU\F|KbgsT |+MHrW¢:[d\aV6 TXHKM`WaVUMHKM6W¢i[]d[d
(46) R bbMRbYMDh`|iqbW-TXHK\ad\aeF[dYDWftTXVXWa[]Rb[]RKe2qr\F[]R:TXgDzR`TUHKM@Y\FRFTXVXWaVU`£¤¥Pn¤ RbWa[]FM ¡IW¢FMDgU[dWaR`VUMDLWa[]RrgWa]L\FgsTWag M .>Y[dMR:TWag[]TUH>TXHKM1\aVX[]eF[]RrWfrTUV Wf[dRK[]RbegUMTDGIHbM|bWFh>q+MVU\aVXL`WfRbYM \a«hbWa¡\i\FgsTWFgoW^MDWfTUKVXMgUMdMDYZTX[]\FR&gXY HKMDL^MY\aL|K[]RbMDh&[TXHTUHKM2RbWf[daM¡IW¢aM6gs[kWfR&htML\aRrgnTXVXWfTUMDg TXHbW-ToRrWf[daM1¡W¢FMDgU[dWaRWa]\FRKMYWfR&RK\aTIHrWfRbhtdM1TUHbM b[]qbqrM6hTXVXWa[]Rb[]RKekWf|+Mkg G \1|rMIWa[]V/[]TUH|+\:\:gnTX[]RbebcaMIL`WahtMW\aKVUTUHVX\aKRrh2\arMtq+MVX[]LMDRFT g[TXH`«hKWa¡\i\FgsT !GIHKM MtqrMDVU[dLMR:T2[kgTUHbMgXWfLMWag2|rM\aVXMac|KtT^[dg2VUMDqrM6W-TXMDh\aV-WfVX[]\Fbg-WfdKMDg2\fTXHKM>VXMeabdWaVU[d§DWfTU[d\aR qrWfV WfLMTXMV C GIHbMMVXVU\FVVXWfTUMDgIeF[]FMR&[]RT Wf|KdM2}Y\FVUVXMDgUq+\aRbhTU\TXHKM C dMDWFht[dRKeTU\TXHKM|+MDgsToTUM6gnT MDVUVX\aVV W-TUMF "$& M2HrW¢aMqKVXMDgUMR:TXMDh¥[dR¥TUHb[dg1WfVUTU[kYdM2Wgs[dLqK]M^gUY HbMLM\FV|K[dRbWfVX&MDWfTUKVXMgUMdMDYTU[d\aR¥[dR¬W>Y\aR:TUM:T \aYdWFgUgU[+YW-TX[]\FR£\aL|K[]RbMDh¬[TXH W&RbWa[]FM`¡W¢FMDgU[dWaR¬YdWFgUgU[rMV6c²TUHK\:gsM^MDWfTUKVXMDg1dMDWFh¬TU\qrMDVs\FVs_ L`WaRbYM6gWFgIea\i\thWFgTUHK\:gsM1\a|KTXWf[dRKM6h>[]TUH&TUHKMYdWFgUgU[kYWf²«hKWF¡\i\:gnTI\aRYdMDWaR&hKWfTXWbcKWfRbhTU\|+MTsTXMV q+MVU\aVXL`WfRbYMDg\aR¦Rb\a[kgs¦hKWfTXWbcMaMDRªY\FLqbWfVXMDhTX\&WVXMeabdWaVU[d§M6h¦«hKWa¡\i\FgsTHb[dY H¬VUM Fb[]VXMDgTUHKM TXKRK[dRKe`\fW^VUMDeaKkWfVX[]§6W-TX[]\FR>qbWfV WfLMTUMV6 GIHKMgsTUVX\aRbeq+\a[dR:TXg\a/£¤¥Pn¤ WfVXM reg. . PQPSR#TJU VU4W. 60.
(47) yDw. . £ kWagXgs[]bMV G V Wf[dRK[]Rbe^MDVUVX\aV G MDgsToMVXVX\aV £¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR ¡W¢FMDgU[dWaR 5.06% 1.95% «hKWF¡\i\:gnT ¨ \aqtTX[]L`Wak© 3.8% 3.06% «hKWF¡\i\:gnT 0.58% 6.33% ¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR ¡W¢FMDgU[dWaR 9.47% 8.59% £¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR uqrMDVXYMqtTXVU\FR 7.36% 9.32% «hKWF¡\i\:gnTMDWfTUKVXMgsMD]M6YZTU[d\aR ¡W¢FMDgU[dWaR 10.28% 9.46% ¤¥Pn¤ MDWfTUKVXMgsMD]M6YZTX[]\FR uqrMDVXYMqtTXVU\FR 11.53% 13.12% G!Wf|b]M} QVUVX\aV!V W-TUM6g![TXH MDWfTUKVXMDg!\FR2TXHKMIRK\a[kgs1TXVXWa[]Rb[]RKe1gUMT/HK\FgUMdWa|rMDdg!HbW¢FM|+MMDR b[dqKq+MDh []TUHqKVX\a|rWf|K[d][]Tn ; 50 P To[dgIVX\a|bbgnT6cKWagI[]ToVUM :K[]VXMDg\FRK]MDgsTU[dLWfTU[d\aRbgI\a!ht[kgsTUVX[]|KKTU[d\aRbgI\a TUVX[dqK]MTXg\a |+\i\adMDWfR&-WfVX[_ Wf|KdMDg [TqKVU\-i[khtMDgWeF\i\ihTXVXWFhtM\ -ª|rMTnMDMRVXMDhtKRrhKWfRbYWaRbh&[]RbhK[]i[khtbWf qr\-MV\agUMdMDYTUM6hM6W-_ TUKVXMDg [TgUMdMDYZT gM6W-TXKVUM6gTXHbW-TIYWaR>|rM1Y\FL2|K[dRKM6h`[TXH`TUHbMRbWa[]FM¡IW¢aM6gs[kWfR>YdWFgUgU[rMV6caHK[kY H>[kgWaR MtqKd[dY[TWaRbh&VU\F|KbgsTIgXY HKMDL^MF MaMDVsTXHKMdMDgXgDcr£¤¥Pn¤ gs -²MV gVX\aL TXHKVUMDM1LWa[]R¥htV W¢|bWFY ig P To[dgobgUWa|KdM@\FRKd>[TXH¥W2rRK[TXMgsMTo\f!|K[]RrWfVX`MDWfTUKVXMDgDcKWfRbh&VXMDgsTUVX[dYTUM6h`TU\>YdWFgUgU[]rYWfTU[d\aR [T1\aVXigoKRbhtMDVTXHKMWFgUgUKLqtTU[d\aRTXHbW-T1htMq+MRrhtMRbY[]M6g|+MTnMMDR¥MDWfTUKVXMDgYWaR¦|rM^YWaKeaH:T|i ]\i\F:[dRKe&W-T@Y\aKqb]M6g\a¢WaVU[kWf|b]M6gcrTUHibg[]eFRK\aVX[dRKehtMDqrMDRbhtMRrY[dMDg@|rMTnMDMR¥TXVU[dqKdMT g\FVkWfVXeaMDV WfL[d][dMDg\a M6W-TUbVUM6g [T goY\FLqK]Mt[TnYDWfR&|rMWaR[kgXgsKM1\FVaMDVU`kWfVXeaMWfL[d][dMDg\a M6W-TUbVUM6gWfRbhTUV Wf[dRK[dRKegsMTXgD \fTUM TUHbWfT!«hKWF¡\:\:gnT!gs -²MV g VU\FL gU[]L[ddWaV MDWa:RbMDgXgsM6g ²[]T!\aVXtg\aRKd@[rhKMq+MRbhtMDRbY1|+MTnMMDR TXHKM2MDVUVX\aV g\a!TXHKMYdWFgUgU[]bMV|KK[dT1gs\WfVWaRbh¥W-T@]M6WagsT\aRbM2\f/TXHKM2MDWadMDWaVURbMVYDWfR¥|+M^gUqr\aTsTXMDhc WaRbh[T goY\FL^qb]Mi[]Tn>[kggs[dL[]kWfV6 GIHKM\aVUTUHbY\aL[dRKeª\aVXtg[]dIWahKhtVXMDgXg2gU\aLM\fTUHb\FgUM>[dgXgUKMDgD $b\aV^[]RbgsTXWaRbYMFc!TUHKMWfqKqbVU\:WaY H Y\aKkh¬qKVU\F|bWf|b]¥|+M`MiTUMDRbhtM6hªVX\aL |K[]RrWfVX¥MDWfTUKVXMDgWaRbh YkWagXgs[]rYWfTU[d\aRªTU\¥Y\FRFTX[]RiK\Fbg@MDWfTUKVXMDg WaRbhVUMDeaVXMDgXgs[d\aR>|i&Y\aRrgs[khtMVX[dRKeqrWfV WfLMTXVU[kYhKMRbgU[TnL^\thtMDdgD ,- I ¡IW-T xf
(48) w " pb¡IWfTsTU[]TU[= gU[dRKe`L2tTXbWf []Rt\FVUL`WfTU[d\aR>\FVgUMdMDYZTX[]RbeM6W-TXKVUM6gI[]R¥gUKq+MVX:[kgUMDhRKMKV Wf RKMTo]M6WfVXRK[]RbebPSR
(49) J #J
(50)
(51)
(52)
(53) 4 ctF\adKLM ;KcyDxFxfwr ¡VUM6~a
(54) ~ " 1¡VUMD[]L`WaR (i\FL^M[]RKbRK[]Tn¬TXHKM\FVU¬\aVqKVXMDht[kYZTX\aV gMRrgsMDL2|KdMDgDªG MDY HKRK[kYWapMDqr\FVsT ; 0 xKc MqbWaVsTXL^MDR:To\f (:TXWfTU[kgnTX[dYDgc RK[]FMV gs[]Tn>\a/£Wfd[]\aVXRK[dWbcK¡MVXaMdMFcK}f~a~F~K reg. . . "$#P"&%.
(55) y4;.
(56)
(57) . ¡ xa
(58) " @¡ ¡\FRKRKkWfRbhKMVWfRbh « (²I¬MD[]eFMRbh(tMdMDYZTX[]Rbe [dRKqKtT¥-WfVX[dWa|KdMDg>rgs[dRKe L2tTXbWf [dRt\aVXL`W-TU[d\aR(WaRbh RK\FRKqbWfV WfLMTUVX[dY`htMDRbgU[TnMDgsTU[dL`W-TU[d\aR ¥PSR + # #2) . c yDxFxab £ (tG~F#~ " oMD]d\ £HbVU[kgnTX[dWaRK[WaRb
(59) h F\aHKR (iHrW¢M_ G!W¢i]\FVD N:?<
(60) #* < J
(61) ? <
(62) ? #%
(63) ? 4 #?< J4 £WfL|KVX[dhteFM oRK[daMDVXgU[TnlVXMDgXgc }f~F~a~b £GxKy " GIHK\FLWFg¤ t£\-aMVWaRb h a\-^«:GIHK\aL`WagD N)
(64) %
(65)
(66)
(67) []dM:_ PSR:TUMDVXgXY[dMRbYMacy6xaxKyF !/!0
(68) N " p rhKWWaRbhl / WfVUTD!
(69) ?7 # ? "
(70)
(71) O # % # ##4? ! F\aHKR¥ [ddM% $ (i\FRbgcy6x 0 NK $!£ ~a
(72) ~ " G1' (²B $KKVXMF c 2o£VX[kgnTX[dWaRK[]Rb[Bc 2 -²ac u¡MDhKRbWfV gsi[=c¤ ' (tY HiKLLMV6cWfRrh / WfbgXgU]MDVD (iKqKq+\aVUT!FMDYTU\aV L`WaY HK[dRKMYdWFgUgU[]rYWfTU[d\aRWfRbh2-Wfd[dhKWfTU[d\aR\arYWaRbYMDVTU[kgXgsKM gXWfLqKdMDgIbgU[]Rbe^L[kYVX\FWaVUV W¢MtqKVXMDgXgs[d\aR&hKW-T WK' &
(73)
(74) %
(75) c y6 ¨ y6~F©cb}f~a~F~K $Q 1~b y " $ $ dMbVUMTIWaRbh J @ML`WfR/£\FWaVXgUM_=TU\a_BbRbM:[kgUbWf²gsMD]M6YZTX[]\FR ? # #
(76) )(
(77) ?
(78) ) 3*+ % ?
(79) cKwry ¨ -y ,f}F© ;+8²yD~ 0 cF}a~a~KyF $Q 1~:4} " $ $!dMKVXMT@WfRbh @MDL`WfR' $rWagsToWaYM2htMTXMDYTU[d\aR¥[]TUH¦qKVXMDY[dgUMq+\FgUMMDgsTU[dL`W-TU[d\aR PSR +.
(80)
(81) )'
(82) 0/2133-1ac+a\F]bL^MyFctqbWfeFMDg}fN ;+8t}aN bci}a~a~F}K $ / G~F#~ " r / 3 $KVX[]M6htL`WfRcG1 / WFgnTX[]MFc WfRbh p/GI[]|bgUHK[dVXWaRK[=«hKhK[TX[]FMd\aea[kgsTU[kY>VXMeaVXMDgXgU[]\FR ^W gsTXW-TX[dgsTU[kYWai[]MD \a |+\:\:gnTX[]Rbeb N #QD4< # c+ } NaN 0 8iwF~ 0 cF}f~F~a~b $3(txaF#W " 5B $bVUMDKRbh WaRbh p o'(tY HbWaqK[]VXMa /tqrMDVU[dLMR:TXg[TXH W RbM |+\i\FgsTU[dRKe Wfdea\FVU[]TUHbL& PS6R + 728:9
(83) O?< ?
(84)
(85)
(86) 3
(87)
(88) 9 <;Q
(89) ? cqbWfeFMDg&y w 48²+y ;fb ¤¥\FVUe:WfR &WatL`WfRKR cyDxaxFK $3(txaa| " 53 $bVUMDKRbh WaRbh(p oL(tY HrWfqK[dVUMF 1WaL^MTUHbM\aVXac/\FRt_ ][dRKMqbVUM6ht[dYTU[d\aR WaRbh |+\i\FgsTU[dRKeb PS=R 7?>
(90) @ N A7!
(91) 7
(92) * *+ B7C; #?< J# c/qrWfeaM6gNF} ;+8tNaNF}K«£¤ lVXMDgXgc MD 5\aVX²c ?5c yDxaxFK ! ~F#N " PngXWf|+Md]M @KF\aR WfRbh «oRrhtVUj ][kgUgUMM -! «R []R:TUVX\thtbYTU[d\aR TU\ -WfVX[dWa|K]M¥WaRbh(M6W-TXKVUM gUMdMDYZTX[]\FR (# ? # ) . E;Q
(93) ? / ? # crN dya+y ; 0 8²yay :}tcF}a~a~FNK 1PnGx:4} " lQ 1 WfRKeF]MDacuPS|rWKc WaRb6h &GIHK\aLqbgU\aR>«R WfRbWa]tgU[dg@\fI|bW¢FMDgU[dWaRªYkWagXgs[]bMDVXgDPSR +.
(94)
(95) )F ! F ' G> 1FcKqbWfeFMDg}a}a4N 8K}a } bcby6xaxF}K ¤¦¡I¡ $~a#~ "1:¤¦WagU\aR c rF¡IW-iTUMDVDcal!i¡IWfVUTUdMTUTDcFWaRbh`¤ $KVXMDWfR ²¡\i\FgsTU[dRKeWfdea\FVU[]TUHbLg Wag/eFVXWFht[]MDR:T htM6gUYMR:TDPSR #< # # # +?? H4 I cKqbWaeaM6'g ;ty64} 8;Ky Ki¤¥PnG lVXMDgXgc }f~F~a~b ¤¥l ~F#~ " &\fmn[¤¥[]:WfHbWaVXWWfRbh¬¤¥[dY HrWfMJ r²lWa§§6WfRK[=2£\addWa|r\FVXWfTU[daMb]TUMDVU[dRKe&[TXH¦TXHKM`gs[dLqK]M |bW¢FMDgU[dWaR¦YkWagXgs[]bMV6oPS@R " / ?< ?
(96)
(97)
(98)
(99) #K N? " # ?< L crqbWfeFMDgI 0 4x 8i axbc:}f~F~a~b . . +. PQPSR#TJU VU4W.
(100) y6 o\-t:}4" poz@¤¦x
(101) " p\:g ; #" G \aV ~a#N " . . « \-:[da\ -! zR&Y\FR:FMVXeaMDRbYMIqKVX\i\fg \aVqrMDVXYMqtTXVU\FRbgPSR ? -F 4? #2
(102) I%
(103) # J# D! N - # ctqbWaeaMDgIby+;+8tF}F}tc+yDxa:}t @KRKRrWfVpoWfTXgXY HcG Wa-WagUHK[@zRK\thKWbcIWaRbh &1dWabgs_ p\F|rMDVsT>¤¥b]dMV6 poMeabdWaVU[d§[dRKe Wahi_ Wf|+\i\FgsTDPSKR +
(104)
(105) ) ctqrWfeaM6'g ;fa#w 8; 0 ~bcbyDxFx b $KV WfRb@p\:gsMDRi|KdWfTsT6FGIHKMqrMDVXYMqtTXVU\FR «(qKVX\a|rWf|K[d][kgsTU[kY/L\ihKMF\aV![]Rt\FVUL`WfTU[d\aRgsTU\aV WfeFM WfRrh\FVUe:WfRK[d§DWfTU[d\aR>[dR&TUHKM|bVXWa[]R' J
(106) # / /cb ; N F4 8:w:~ bcKyDx ; K &WfVX[²G \FVUia\FdWb $KM6W-TXKVUM@MiTXVXWFYZTU[d\aR|:`Rb\aRt_ qbWfV WfLMTUVX[dYL2tTXbWf²[dRt\aVXL`W-TU[d\aRL`W-t[_ L[]§6W-TX[]\FR (# ?
(107) ) . E;
(108) ? / # cbN ]yDwb4y ;+8+yDwFN KcF}a~a~aNb . "$22! 2 D ´ µ !f ´! }K ]#y bD \fT aW-tTU [d\a!´ Rrg @1111@1111@1111@ }K } lMDVXYMqtTXVU\FR&WaRbhRrWf[daM¡IW¢aMDgU[kWfRYdWFgUgU[rMV 11@1111@ }K N «hbWa¡\i\FgsT @1111@1111@1111@
(109) Nb]¯ y#K PSRK ¯ \aVX L¯ #Wf¯ TUa[d\a R ´ TUHKMD!\abVX`¯ TUµ \i\F!´ dg¬ ´ 1µ @ ´ 1 # 1 !1 # 1 ´ @ (1t !´ 111@ Nb } £\FRbht[]TU[d\aRbWa¤¥tTXbWfPSRK\aVXLWfTU[d\aR¤¦W-t[dL[]§6W-TU[d\aR 1@1111@ Nb N ¤¦tTUbWaPSRt\FVUL`W-TX[]\FR&¤ªW-t[]L[d§DWfTU[d\aR 1111@1111@ Nb w £\FLqK]Mt[Tn 1111@1111@1111@ wr]y ¯ W- T W`¯ gsMDTX g @1111@1111@1111@ wr } hteaM1M6W-TUbVUM6g 1111@1111@1111@ wr N G V Wf[dRK[]RbeWaRbhTUMDgsTU[dRKe011@1111@1111@ ;K]y¯ #Di q+MVX[dL^MDR:TXg[]TUH¥WaYDYKV W-TXMTXVXWa[]Rb[]RKe`gUMT g 11@1111@ ;K } iq+MVX[dL^MDR:To[TXH&Rb\a[kgs`TXVXWa[]Rb[]RKe`gUMT g 111@1111@ !´ # ! ´. . N w. . 0 . . . x x. yF y yFy. . "$#P"&%.
(110) Unité de recherche INRIA Rocquencourt Domaine de Voluceau - Rocquencourt - BP 105 - 78153 Le Chesnay Cedex (France) Unité de recherche INRIA Futurs : Parc Club Orsay Université - ZAC des Vignes 4, rue Jacques Monod - 91893 ORSAY Cedex (France) Unité de recherche INRIA Lorraine : LORIA, Technopôle de Nancy-Brabois - Campus scientifique 615, rue du Jardin Botanique - BP 101 - 54602 Villers-lès-Nancy Cedex (France) Unité de recherche INRIA Rennes : IRISA, Campus universitaire de Beaulieu - 35042 Rennes Cedex (France) Unité de recherche INRIA Rhône-Alpes : 655, avenue de l’Europe - 38334 Montbonnot Saint-Ismier (France) Unité de recherche INRIA Sophia Antipolis : 2004, route des Lucioles - BP 93 - 06902 Sophia Antipolis Cedex (France). Éditeur INRIA - Domaine de Voluceau - Rocquencourt, BP 105 - 78153 Le Chesnay Cedex (France).
(111). . ISSN 0249-6399.
(112)
Documents relatifs
À cet effet, un nombre considérable de recherches démontre cliniquement que les enfants de parents souffrant de problème de santé mentale sont plus à risque d’un retard dans leur
usual route to school and provide us with some details about their travel routine (e.g., schedule, mode of transportation, people accompanying them, etc.); 3) recent interventions
La question de l’extension des limites de Paris jusqu’à cette ligne avait été discutée mais ne fut tranchée que par un décret impérial du 9 janvier 1859 qui décidait de
In this paper, we study the performance of an enhanced channel estimation technique combining estimation using an autocorrelation based method and the Expectation-
Selon Laeven et Valencia (2012) , le coût budgétaire de la crise bancaire s’est respectivement élevé à 44 et 41 points de PIB en Islande et en Irlande, ce qui place ces deux
Dans ce travail, nous comparerons les entreprises coopératives, détenues par leurs membres, dont le produit risqué est la ressource critique apportée et dont l’objectif est
L’indice EMBI a augmenté de 77 points de base entre le point bas de mi-septembre 2007 et fin février, en partie conduit par la hausse de 300 points de base du spread de l’Argentine
