!"#$ %& '( )*+,&-.+/0 %& 12+/+&-.
3 454 678 /90: 8 /+;<&. 44 =8> 2-8 /2+-& %& 78 /90: 8
/+;<&. &/ ?@@'+A
8
/+2*. %& 12+/+&-.
B 5 CD5 E5 F
A+&*A&. 12<- '(G*H0*+&<- I ?0-2*
8 </+;<& F 1GI? 4B4 B JK LMNOPQR STU V 45K WXXY Z 6 [\]^]_`\]ab cd `eefaghi_ efaj`j]^]_\i_ j`_ki_ _lf ^i_ gf]\mfi_ ib\fae]nli_ ealf ^` fighifghi cd]boafp`\]ab _lf _leeaf\_ pl^\]pkc]` E5B qrB6 78 -A ?-* 8 <%2*s t9-+./+ 8 * u'+,+& -v w5 qrB6 u'+,+&- ?' 8 / 8 xyz{ |} ~Z } 3 }~ ZZ ¡¢ £ ¤ ¥ K }¦|§~ ¡¨ ©¢ £¨ £¨ ªª ¥ K } E E5 qrB v } ~§ E5 qrB }~Z ¡¢ £ v
v CB 5 B JCK 5 KB5 K5 B BB J 54 B ¥ B5 } K KB B4B B B KB 5B 5 v qB } C BK B 44 5 ¥ 5KK K K 5 5q5q BB ¥ 5KK4B B5JB } ~ B J4B C45 C4 BB J CB5 K 54 K B 5qK BB JCB 5 B54 } | 45 ~ BB BBJB 5 3 BB B K K B KC Cq C55 C B w B K }~ B 45qB JC B 5 ¥ 54 5 5B B B JC K 5 KB5 K 55 5 B ¥ 5B qB B 5KK4B ¥ JB BCBB } 5BB5 4K B w 4K ¦ B w 4B K B5J 554 5 K } K B 4K K5 5B BBB Kw q4KJB B Z BB | B 5 ¥ J KC KB C 5 K BrK 5 5 K4 B4 4B BB KBB B C B BK K B55 J4 } 5BB5 4K B B B BB J5 w 5 5qK 5 K 5q BB BC B } K 5Bw 5 445 | B BB qBB B BB 5qB B B 5w B B KB5q BB 5 Z B 4B5 KKJ BB 4B qKB 5qB B | B 54 K K4 B5 J B4 544w | B 5w BBJ 6 BBw J B 5B 4 B BJB 55 ww 54 BJC K K 5 B B 44B4 KB C4B B4BB J 5qB B } EB B B B5 KB BB K 5B KB v 4 C554 Kw q4KJ B BK 44 B BB B 55r q q 4B BB BB B J C4 CB J § q5 BB rK 4 } ~ Kq4KJB B 4 r B Kw q4KJB B 55 K 5B 5 BJB C B KB 54B 6 v 4 q qK BB ¦ 4 5 3 K Z 5 BK K 4qr BB J B KBJB BB5B 5B C B K KB BB 5B v 4r 3 K ¦ B B ¥} C 4w K B4 5 q | 4 q | | J BB BB rB BB J 45 5 5qBB 4B B B ¥} C 5B C 5 B B B BB J JK K rB B 5J B 4J J KB 4B Kq4KJB C }
C B B B J KC 4 K BB q ¥ 5KK5 4r C K B J B B44 B BB } B K 5B B 4B KKB JB qKBB B B 5qBBB KB 4KB B 5B B B 55 K K B BB } ~ C B B B } B 5 w B J4B C K C KB 44 4KB 5B 5B WV rB 4B } K K4K B KB ¥ CB B5BBB 4wrB Bw5qr B J B 4B KK4B B 4B5Bw B 5 5 45B BB 5BB } 5BB5 4K B B B ¦ K ¦ 4K KB5B B 4B BB4 Kr B ¥ B K B B X } K5 4K EK 5 BKCB55 C4w BJ B 5qBB KC 4 BB B B 4B } C B4 B KB J 5K4 KB JC B BB B J KK 5KKJ 6 q 5 ¦ 4 } C 5 Cq 4B 5 qB 6 B 5 C B } 54 KK B J B ¥ ¥ 4B JJB B BK J BB q B } 3 B BB 5K B C45 5B K5KB5 5 5qJBBK5KK BB4B B K BKB w B B B 55 B KB r5B 5 6 4 K 5qB C }~ B B B KwK4 4 5B 5J BB KB C BK 55B C w K K K C BK rBwK BB B JJ B B 5KK5 rB K } C 5B B 5 KB r5B 5 5KK B Cq B J 45 K B 5 JC 55K } K B CB 4 5 KB5 K r 5 J JC 44 BB KB B 5K rB }
~ B rKB B45 KrB B B 5KK B K KB 5B B J 5KBB 4B K B CK } B B B B B4B 4B 5B rKB B 4B BBKCJ44 KKB 45rCK 5r J 4B4 } ~ 5K 5 qrB B CBC 5r 5 rK B45 KrB JK BB C 5 ¥ K B } ~ B5 B C B K Kq4KJ } B B B rK 5BBJ 4K C C 4BB } ~ 4BB BB K 45 B qKJB 5BC B4B 4K 4 } 5B rKBB w B 5r CB B B4 KKB 5 4B CBK } ~ B 5qB KB K4B B B ¥ B5B5KK B B4J5B B B ¥ B C K } B B 4BBB rK 4K C C 5q } EB 54 5 rK B 5B4B 5 44 CBw K C B4 qBKK } EB 5B ¥ KBBBB CB C 5r B B 5 ¥ JB B JB K BK 5 B B B 4K B 5B 4B KB C 5 4B B K4qB B CB B 5rB CK } v B K4qBB 5KB4B 5K ¥ 4 CB K J BK4qB 54B qK K B545B Kr }
B5 KB J 5 w KBB5q B5KBBBK5BB } q KBB B B qB KB B J KK 5 K 5 q 5 } Z q 5 B qB qBB B q B B5q 5 B K B5 K 5 B B 5BB 5 ¥ }Z q BqB5 KB q B KqK5 5 } Z qB B q 5BB5 K q K q BB } B ¦ BBBB ¥ 5 5 qK5B B B } qB BB 5 q B q5q B B q KK q 5B B K q BK } 5qB K KB B5 BB B5q B BJ5B B5 KB } 5B q Kq B5q 5q } q 5 B q K B q BK B qBK } qB KB B q B 5 5 B q5q BK K q KK EB5 ~ q 5 } Z qq qB 5 KB B KqB B K 5 }Z qB KqB5 5KB BK 5Kw ¥ B q B KqB qBB 5B B5 q K }
v 5q5q B B4 5 C5 B Kq4w KJB BB 5B K B } 5 44 54 B 5w5 B 5B 6 5 BB ~ q4KJB 5B CB4 B v qB BB ~ 3 KB v KK5B CB4 B K 545 ~ 3 KB v KKw 5B CB4 B ~ B 5 ¥ B C4 5 K4K qrB B B 5rB Cw KB B4B B 4B rKB B45 KrB } B 5 4 CB 5r B 55r 4 JK BB C rK K CK } B 5 45 J 5w r B K ¥ 4 B C J C554 C 5r K B C K K ¥ 45 } v K4K qrB B B4 5qB } EB 5q C5 B 4BB rK B 55B 5BBJB C CBK BB K ¥ KK BK5 K 44 B rKB B45 KrB J B B B } B KB 5 54 5B rKB 6 BwK4B CB K4q K ¥ KK BK5 } ~ 5q W B 5B54 C5 B 5rB CK B 444 } B 45B C4 5J K rK BK4B } ~ K 44 B 5rB CK J B w BB B B B 4 } B BB B K4qB 5KB CB B 5rB CK } v B K4qB B ¥ K4qB 54B 5BBJK4B 5K ¥ 4 CB K 5B B5 45B Kr B B 4B4B B W } } 5q W }~ B 5qB V B 5q5 5B54B rK 45B B45 KrBK4J B J B B 5w rK 4 5B B 4B } ~ 4K C C 5BBB 4BB B qBJK K 5B rKB BJ 4 B 5r } B 4B 5 5 ¥ B 5q } ~ B K4qB 5KB B B4B B5 B 45B Kr B 5B KB B 5B KB B KB ¥ B }
~ B 5qB KB 5q C C4 C 5r B5 B 4 K 5 } v 5r BCB5 B 4B ¥ BBB 5K ¥ 4 B5qBJ BY 5 E ~ | Y ¦ XV } EB 5q V B B 4BBB rK CBK B4 qBKK } B B B ¥ 454B B ¥ 5qB KBB 4BB 5r CK B5 B B KKB C 5 J 4 rK 5q ¥ CqBKK } B B 5q rK 4BB BB B 5B B B 4BB B 4B } v 5 ¥ B 4 K B | | 4 | XW | X } B B B 44B B Kw JB CBK B K4qB 5KB BJC 4 5KK BJ 5 BK BB K C 44 5 } ¥ B 5Kr 5 K4K } B 4BB qK w KK KJ ¥ B4 B B rKB B45 CqBw KK 5q V } ~ B ¥ B | v 5 B 5B v v 4B B B 5KwB 545B K B } 5 BKB Cq 545 C r ¥ E }
4BK4 } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } B5 } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } 4B 44 } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } Z B KrB } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } G*/-2%<A/+2* } 44 J } } } } } } } } } } } } } } } } } } } } } } } } } } } } } W ~ B45 KrB } } } } } } } } } } } } } } } } } } } } } } } } } } } ~ BK4B } } } } } } } } } } } } } } } } } } } } } } } } } } t-+/-&. %( +*2-: 8 /+2* W } 5 ¥ BB B 5rB } } } } } } } } } } } } } } } } } } } } W } W B5 C 5r B 6 5 C } } } } } } } } } } } } } W } ~ 4B } Bq } } } } } } } } } } } } } } } } } } } } } } } } } } V W } v K ¥ 4 CB } } } } } } } } } } } } } } } } } } } } } } } } } = &. :2%'&. 8 </2 -0H-&..+. H 8 <..+&*. } E4K C } } } } } } } } } } } } } } } } } } } } } } } } } WW } W E4K B } } } } } } } } } } } } } } } } } } } } } } } } W } ~ 5B KB } } } } } } } } } } } } } } } } } } } } } } } } } } } WY = &. A9 8 *&. %& 78 -2, :<'/+@'&. } v B 4B } } } } } } } } } } } } } } } } } } } } } } } } } } } V } W ~ B 5qB KB } } } } } } } } } } } } } } } } } } } } } Y } v qK4J } } } } } } } } } } } } } } } } } } } } } } } } } } } X } v qK4J } } } } } } } } } } } } } } } } } } } } } } W } V ~ C44 J BB } } } } } } } } } } } } } } } } } } } } V } E4K C C v } } } } } } } } } } } } } } } } } } =8 .0'&A/+2* %(9+./2 H-8 ::& V } B rK } } } } } } } } } } } } } } } } } } } } } } } } } } } V V } W ~ 5 BB B 4B } } } } } } } } } } } } } } } } } } } } V V } BK 5 } } } } } } } } } } } } } } } } } } VV V } 3 KB } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } V V } V 5 5BB5 } } } } } } } } } } } } } } } } } } V V } 5B K CK } } } } } } } } } } } } } } } } } }
8 <..+ 8 * -&H-&..+2* } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } W B } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } X } K 5 q B } } } } } } } } } } } } } } } } } } } } } B K5 B } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } V 3 KB } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } YW } 3 q BB } } } } } } } } } } } } } } } } } } } } } } } } } } } } } Y } 5 J q B q 5K B5 Kq X ?@@&*%+A&. ? )* 8 'H2-+/9:& %& @-2H -8 :: 8 /+2* %* 8 :+;<& t2% 8 H& 8 -+/9:0/+;<& @2<- ' 8 %&.A-+@/+2* %( <*& %+./-+ > </+2* t G*2-: 8 /+2* t-+/&-+ 88 *% 8 -+/9:&/+A A2%+*H. 8 * +''<. /-8 /+2* 2* -8 +: 8 H&. =8 -&A2H*+/+2* ,+ 8 9+./2H -8 : >8 .&% &./+: 8 /+2* + > '+2H -8 @9+&
ON Q TNUMQ EB 5 B 4BB B 55B 5BBJB C CBKw BB K ¥ KK BK5 } b\fae]i ~ C B 54 BB54 B 4 } B B rK 4 JC 5 CK 5 } EB 5K4KC B KBBB BJ B5
Ω
B B5 KR
K | KB4 4 B4f
6¯
¯
¯
¯
¯
¯
¯
¯
H(P ) =
−
X
x∈Ω
P (x) log P (x)
H(f ) =
−
Z
R
f (x) log f (x)dx
} BBw B J0. log 0 = 0
} ~ B qK CKw 5 } ~ C B K B4 B B KB B J qK KJ CBq BJ } 3 5 4 B KB 4B 4 B } BB B5 B5 5m
B K B } K B B BB B4 5 B[0, 1]
5CB B K J B } K B B BB B4 5 BR
K 5 ¥ 4Bµ
σ
2
5CB B KN (µ, σ
2
)
J B } v CB 5 BB J5 B C B 5 B K Cq qrB J 5 B B C 5B B } EB 5B 5 K BB B 4B 4 B B }W b\fae]i gfa]_ki v B4B K ¥ BB 4BB K K B5 w B4 5 ¥ 4J4B } } B 4BBB B K4J 5B ¥ BB4 5B4
¯
¯
¯
¯
¯
¯
¯
¯
H(P, Q) =
−
X
x∈Ω
P (x) log Q(x)
H(f, g) =
−
Z
R
f (x) log g(x)dx
} W 5KK4 B B B B5 B 5B B5 5 B B BBP
Q
} ~ C44 5 ¥ 4 B 4B B 6 1-2@2.+/+2*H(P )
H(P, Q)
H(P )
≤ H(P, Q)
v 4B 4K 4 44 CK 3 q 5B B KB q4 CK B 3 qY } K 4K 4 CK B K4J 5P
Q
5KKK(P, Q) = H(P, Q)
− H(P ).
} v J4 B KB 5 4 C J 4B BQ
P
} v CB C B KBB 5B K KB C45 ¥ B 4 }| BB | B 4 B 44K K BΦ
w5B CK 5 } B B B 5B B C C 5B4 5B 5q B J B KB 4 JCB 5 q4 5 } _\]p`\]ab `l _ib_ cl p`]plp ci f`]_ipj^`bgi B 5 B5 BBJ(Ω,
A, P
θ
, θ
∈ Θ)
BP
θ
B B Bw B ¥ 4B CBKΘ
} v B5 B5 5B 5 5B BB J 5qJ 4P
θ
B4f
θ
KB ¥ B 5BP
θ
f
θ
B B 5B 5 K CB } ~ B 4B BB5 B B B 45qx
n
= x
1
, . . . , x
n
n
4K B4 B 4 5P
?
} ~ C45q 4 4 5 BK5 BΘ
→ R
θ
7→ P
θ
(x
n
)
} B 5B 44 } B 5q B 55B C B B 4w BBB CB4 wBK5J B Bl
6l : Θ
→ R
θ
7→ − log P
θ
(x
n
).
} V~ K4q K ¥ KK BK5 45B 5qBBJC ¥ B B J Kr
ˆ
θ =
K(l(θ), θ
∈ Θ)
} 4 BB54P
θ
ˆ
5KK BKP
?
} B 4B B 5 K4qB B B ¥ B 5BBJB 5KK qBJ § Y 5q B q4J J BCB5 B J K B } ~ BJ B5 BBJ B 5B 45 BK5Θ
→ R
θ
7→
n
Y
i=1
P
θ
(x
i
).
} wBK5l : Θ
→ R
θ
7→ −
n
X
i=1
log P
θ
(x
i
).
} Y ~ B B KB BB Jn
−1
l(θ)
5 B C 5B4H(P
?
, P
θ
)
J 4BBH(P
?
)
5K4K C44 3 q } 5B4J KB B KJKB K4q K ¥ KK BK5 5CB 5qB 4 C 5B4 5P
?
B 5q CP
?
} C B KB5CB 5qB B J B 5qP
?
BB CK 5 } } OQNT Q RT OQ B 4BB 5 rK 5BBJ B45 KrB B 5 BB BK J 5 B 5 K4K KB B 5 54 BJC B 4 5 K4q K ¥ KK w BK5 } v B4B B B5 BBJ(Ω,
A, P
θ
, θ
∈ Θ)
4x
n
4K B4 B 4P
?
} 45BB J C 5 4P
?
CBK BP
θ
C B B 4 } B 4 B 445B B ¥ 5B 5B ¥ K BY E ¦ X J B5K 4BB K 5q KB B 5B Kr 5 4 BYY B 4BB 44 ¥ B 5B 5 } 3 CBBB CKB BB 45BB B BBBBw BB 5q5B K B BKrP
θ
, θ
∈ Θ
BP
?
} BB CB B4J5 X 45BB JC BC B 4BB C B 5B r5B B Bw K Kr B B | 5 BP
?
} 5 Bw B4J5 K 4B B BB BB JC B 45qC 3 Kr B B KB ¥ 5 BP
?
} EB B 5 K4KB JB C 5P
?
KrP
θ
, θ
∈ Θ
B B BB K } C B5 5 45BK5
P
?
B 5 BB 45B B 5B B Kr } 3 5 ¥ 4P
θ
?
4BP
?
5 5 } i_ pacm^i_ EBwB 5B BBwBKBΘ
i
, i
∈ I
Θ
} B BBw KrB KrB } EC Kr 44 B45 KrB 5CB 4K B 4Bx
n
JB BBwKrB Krθ
?
} i
∈ I
¥ 4 B B BJC ¥ Bˆ
θ
i
Kr BK4 BB K ¥ KK BK5 C4Θ
i
6ˆ
θ
i
=
K(l(θ), θ
∈ Θ
i
).
} CB B C B B BB4 J B KrB B K4JB 5 B J 5 K BBwKrB BB K } ~ B 5qB B B KrB B J 5q V B45 CqBKKB B B B 5 wK4J } 7 2%'&. @ 8 -8 :0/-+;<&. * 8 /<-&'. EB 5B K4JΘ =
R
m
5 B Bm+1
BBwKrB K4B{0} = Θ
0
⊂ Θ
1
⊂ · · · ⊂ Θ
m
=
R
m
} Xθ = (θ
1
, . . . , θ
m
)
∈ Θ
j
B BK Bθ
k
= 0
k
≥ j + 1
} v B ¥ K B KrB B4B 4K C C 4BB B 5q } 4K B BBwKrB BB B B ¥ Krθ
0
445 5B4 B2
m
BBwBKB ¥ 4B BS
[[1, m]]
6Θ
S
=
©
θ
∈ Θ
B J∀j ∈ S, θ
j
6= θ
0
j
∀j /
∈ S, θ
j
= θ
0
j
ª
.
} Krθ
∈ Θ
S
B BS
} B BS
?
Bθ
∗
} ~ 5B B 4J Bθ
0
= 0
B45 KrB B 5q5q B 5KBB Bθ
∗
} B BB B 5B KrB B B 45B JCB ¥ KrB K4B } KrΘ
k
4K B 5KBB C5Bl < k
B }v B 5KBB B BK4B B C 5Br BK B KrB } X B JCB B B B 4K B BB B KrB } } M R N NT B B 5 B B ¥ KB BKB 5 54 B45 Kr 6 BK4B CBK BB ¥ }V acm^i ci ^a]_ bafp`^i_ ~ B 4B
x
n
B 5 B4J5 KB 4B J CB BB K4B B B KJB B 454KK B C B 4Bw B 4B C K5 } CB5 B CK K 5 BB B B B KrB B } ~ CB5 B w KrB B 5R × R
∗
+
B B4B 4 Bf
µ,σ
2
(x) =
¡
2πσ
2
¢
−1/2
exp
µ
−
(x
− µ)
2
2σ
2
¶
, θ = (µ, σ
2
)
∈ R × R
∗
+
BK5 C45q C C B BC45n
Y
i=1
f
µ,σ
2
(x) =
¡
2πσ
2
¢
−n/2
exp
Ã
−
2σ
1
2
n
X
i=1
(x
i
− µ)
2
!
.
B Kr 445(0, 1)
K K 54 4 5B4B B BBwKrB ¥ K4B } 6Θ
∅
:
N (0, 1)
Θ
1
:
B BN (µ, 1)
5µ
6= 0
Θ
2
:
B BN (0, σ
2
)
5σ
2
6= 1
Θ
{1,2}
:
B BN (µ, σ
2
)
5µ
6= 0, σ
2
6= 1.
~ rK B45 KrB B B 5 B 6J 5B BBwKrB Krθ
?
w BB 4 5 JB K4q K ¥ KK w BK5B B B C B 5rB CK } B KrBR × R
∗
+
BK4B BB K ¥ KK BK5 Bˆ
µ =
1
n
n
X
i=1
x
i
, ˆ
σ
2
=
1
n
n
X
i=1
(x
i
− ˆµ)
2
.
B Jˆ
µ
6= 0
ˆ
σ
2
6= 1
5 4 } E BS
⊂ [[1, 2]]
5 B KrB(ˆ
µ
S
, ˆ
σ
2
S
)
BK4B BB } B Kr B5Θ
S
K BK5 J(ˆ
µ, ˆ
σ
2
)
} 5B4J BK55 B KrB 5K4 CB K J KrΘ
{1,2}
5 K K B 445 C45q BN (0, 1)
} 5 J K4q K ¥ KK BK5 4r JC BB K ¥ KK 4 B B KrB BK } v q4Kr 4 B 5 54 B45 KrB } acm^i ci ck_ ~ BK4BCBB4B4 ¥ B5B }3 BB 5 B J B 4B 5BB B4J5n
B 4BB5B C 4 } K CB BC B J 4 B 4 } K 5 B ¥ KrB 6
P
np
(X = j) =
1
6
,
∀j = 1, . . . , 6
J 5 5 KrP
p
(X = j) = α
j
,
∀j = 1, . . . , 6
5P
α
j
= 1
J 5 V KrB B } ~ B KrB BK4B BB K ¥ KK BK5 ¥ rK Bb
α
j
= n
j
/n
5n
j
KC B5qj
BB4B }b
P
p
4 5B } 4 CBK BB K ¥ KK BK5n
Y
i=1
b
P
p
(X = x
i
)
≥
n
Y
i=1
1
6
,
C44 B B JK Bn
B K J 5qJn
j
n/6
J 5B B ¥ KrB B } 5BBB K ¥ KK BK5Kr 4 BB B K KrB B B K 5 K K B 4 B 4J4 } acm^i cd]b\ifea^`\]ab ea^bp]`^i v Kr B 5q rK 4BB J B 4 B 4B B 5q } K q4Kr BK4B } ~ B 4B B 5 B BKn
B(x
i
, y
i
)
Bx
i
B 4KBB B5B By
i
B 4By
i
= A
?
(x
i
) + ε
i
, i = 1, . . . , n
5A
?
B K45Bε
i
4Bn
w45qCε
∼ N (0, 1)
} v qBBBBA
0
, . . . , A
n−1
BR
n−1
[X]
45B JC KB =
P
b
i
A
i
BS
⊂ [[0, n − 1]]
B BK Bb
i
= 0
⇔ i /
∈ S
} ~ B KrB B4B } 5K4 BΘ
S
=
{L(A(x) + ε),
B(A) = S
} , S ⊂ [[0, n − 1]].
BJCε
B BBCB4wBK5C45q K KA
¥ 4 B B 5BB rB B5n
X
i=1
(A(x
i
)
− y
i
)
2
.
5B4J BK5 K ¥ K B C KrΘ
S
5B C 5KKB K C 5 K K Kr } B K C ~ 4n
− 1
5 } v B J5 BK5 KrΘ
[[0,n−1]]
J 5 K ~ B K } EB BB 5B BK5 4r K 4n
− 1
KA
?
J 4 }}} ~ K 5 K C BK44 k_lpk EB B B 454B KrBB B KB BK4B K4q K ¥ KK BK5 } v q4Kr B 5KK B K4BB BBJB 4B 6BK5 4r JC 5 KrB BK } ~ B 5 ¥ B B4B 5 rK 4K 5 ¥ BB C4 5 K4KB B 5rB CK } EB 5q W B B 4BB 4 BBB qBJ B 5B } B B 4K B K4qB 5KB 5 BJB B B BB } ~ B 5qB V B 4B4B ¥ 4B 4BB q4JB ¥ 4K ¥ B BJB B B 4 5B 5 qrB }
~ B 5rB CK BB KBB B W } 5 5q B 4B } B J K5 5 5q5q B 4 CB 5r B 55r 4 } CBBBC B B CBC5rwK5 J B 55 B 55B5B } B B 5 5 B 45B C B K ¥ 4B J B B 5rB CK B B B BB 4B B B JC ¥ B BJ 5B JC rKB B45 KrB } 5B 5q B BB B 5rB 5BBJB B 5KK B rK B B45 KrB } ~ B 5qB V 4B B5 CB 5r B5 } B B 4BBB 4K 4B K4qB CBB 5rB CK } ~ K4q B 5KK B4 W } W 4B 4 45BB 5K ¥ 4 44 55B 5 5K ¥ 4 ¥ BJ B B4B B KrB B 45B } }~ CK4Kw B K5q B B 5B KB } B 4BB B W } 5 5q B K4qB B KB 5BB 5 B 5K ¥ B B K5q } NNQ Q Q Q NT Q N Q B B B } W B KrB 5K4 B B
Θ
i
⊂ Θ, i ∈ I
} ~ CBK BB K ¥ KK BK5 C4 C Kr ¥ 4Θ
i
Krb
θ
i
5KK } }~ wBK5l(b
θ
i
)
4 } V 5KK KB C4J KrΘ
i
¥ 4B } B 5q 454 J 5 KB B K 4 rK B45 KrB BJC BK4B } ~ JB J C B B B B B 6 5 KrΘ
i
5 4J5B KB J ¥ ~ C ¥ K C K B } K 5rw K C4 5 JB 6 5 K ~ C 5 ¥ B 4 K ¥ Kn
− 1
K }X ~ C4 B 5rB CK B B B 6BB 5qJ Kr 5K4 B
p(Θ
i
)
K 5 5 C4J Kr ¥ 4B KB4l(b
θ
i
)
} v 5q5q 5KKB B 54 KKBl(b
θ
i
)+p(Θ
i
)
} BB ¥ KW } WJ 5q ¥ K 5 ¥ BB K B B5 CB 5rB } ]_\af]nli Kr B 45B K J B 5rB 44B BBB Cq B 5 ¥ ¥ 4B 5 B } ~ C4 CB BK5 5 45BB B B Kr B B 5 B 4B X 5 B ¥ B B } ~ rK B45 C C 5BBB 4BB B CB B 5rB CK } B K 44 44 3 q 3 q J K B 4BB B KJB B } J 5r B } B 45KK | B | XX BC4BB B 45qB B 4B B J CB B 5rB B B 5BBB 4BBB KBBK4BB KB ¥ 4B B 44 } B X } EB BY Bq 45 B 45B B KJB B KrB 4Bw B 5qBB 4B 5rB CK } ~ 5 B KJ 5B rKB 4BB B 44 Cq 6 | 4 B | XX | XW BC4BB C4 4B B B B 4BB B J | 4 | X ¥ K B B J C BB 4B KBB BJ 5qB } ~ B 5qB KB 55q4B B C 4 5 B K 5 CK } ~ B 5rB CK K w KB 5B 5BBB } v B 44 44 rB B ¥ B C Z Z V B 4 B B K Cq B B qrKB B 5B 5B B ¥ } ~ } EB B w 4B YX BB BY B 5qB KB 5K ¥ 4 B5qBJ JC 45B B 5 44 B BY BY B 45KK B B } ~ B ¥ B B 5 E ~ J B B 5 w BB BY ¥ 4B B 45B | } | Y ¦ ¦ XV } E 5 B 44 5 qK4J 4B4 BB B } v 5qJ B B4 B B rB JC BC 5KBB 55K CKKK B K ¦ WXXX 5KBB CKB ¥ B B B KB ¦ W 5KBB w 4 } ~ CK4K 5 5 B J B rKB J B 4B B 4B } B v Y Y } B B 5KK 5 qK4J B 4 5K ¥ 4 B5qBJ 5KK 5 r 5KK 5r CK } ~ C ¥ K qBKK C B4 4 rKB5rB CK } EB 5 J4BB 4BB B4B B K KB B KKB C B5 5w ~ Y B B 5r X } B 5B BB B ¥ BB } J 5 5K ¥ 4 B5qBJ 5 w rK B 3 W } B 45KK B 4BB wB KJB 4K4 B 5 K KK B ¥ v B v B v BXX J B B B BJ CBK } v B 4K B ¥ | 4 q5 | X J 4 B45 CqBKK C C 5r C 5 B 5K C B K4qB 5KK B } B B J 5B 5 ¥ B 45B 4BBBBKBw K B 5qB KB B45 CqBKK B KB 44B CB 5rB CK } B B B 4 B B 5qB V J B 5B54B } qB 5B KB 3 5q 3 5qYB 5r rB w 5q B } B C4B B 5 5r B 5 K4K B B ¥ ¥ B 45B ~ } ~ X B 4B 55 } ~ B ¥ 44 ¥ Bq } | YY BYY4 B5KKB B KJB B KrB 5qBB 5r } 3 C B 5B 45B w } B B 5r
ϕ
β
B JB B W } } B BB Bϕ
β
5B 5 K4K B 4 ¥ 4 } v B B ¥ | 4 | X J BC4BB B 44B B B BJB 4BB CBK 5rB CK }v 5 CBw B ¥ BB BXBC 5 4K C ¥ K B rKB B45 KrB 5 5q B Kq4KJ5 45B } afpi kbkf`^i ci_ gf]\mfi_ cd ]boafp`\]ab @-&..+2* EB K K 5 J } W B B 5 K 44 B 5rB CK J B 4B B } i
∈ I
B B|Θ
i
|
K KrB B BK C4 KrΘ
i
} 5w r CK v B 4B BB K C 5 v: I
→ R
5 v(i) = 2l(b
θ
i
) +
|Θ
i
|α(n).
W } B J K BK5l(b
θ
i
)
B 4 } V } }~ 5 W 5 K B 5 q } K B ¥ K B5 5r C J B B W } W }~ ¥ rK K B 4 B B B JJB B } EB 5B 5B 5 KB 5r W }n
−1
} ~ K BK5n
−1
l(b
θ
i
)
B BBK C } Bw 5P
θ
?
}v CB 5 J C B5KK B XY K 5r J 4B4 4B 5r CK } B BKBBB 5 KBJBB }W G*/&-@-0/ 8 /+2* ~ J4 v
(i)
W } KB C4J B 4BΘ
i
44 B 5 Kr } 44 v(i)
≤
v(j)
B J KrΘ
i
4B K 5KKB C4J ¥ 4B KB42l(b
θ
i
)
5 5 Kr KB4|Θ
i
|α(n)
JΘ
j
} CB KB v(i)
≤
v(j)
5r K J B 445 KrΘ
i
JCΘ
j
} 5r CK 4 5 rK B45 Kr 5B4 ¥ K 6bi =
K(
v(i), i
∈ I) .
W } W 5qB BΘ
bi
5KK K Kr 5B4 5 B 5 } v K4qB45KKrCB B BBB }~ BJ K KrB 5K4 B 44 45BB 55 5 5rB } B 4BB B W } 5 5q B K4qB w B J K K K 55B 45BBB B45 C Kr } &: 8 -;<&. .<- ' 8 @0* 8 '+/0 ~ W } 4BK4 B 44B BK 54B B B BB 5 K4K } ~ 44 JC ¥ K4 B W } B 4 K KrB B Kr } v 5q ¥ CB B qB B B J B w 5 ¥ 5rB 4B B 5 K4K B 5 K } 5B4J B 44K B 5qB B C ¥ BB B 5rB B 5 4w 4α(n)
} B J K4q K ¥ KK BK5 5 BwK4 Kr44 4 5 q4Kr } 5B4w J B 44 B Kr 5qB W } W BJ 5 C BwK44 } CB4 B B B q4Kr B BBwK4B }~ 5q ¥ 44 B B5 C 5q ¥ B 5 rK 4 B 5 K4K } B 5J B 5B ¥ 5B WXrB 4B B 44B J 4B B 5 44 } B 45B4K B B ¥ v B | } | | 4 v B || | XX | XW | X B KB 44B Kα(n) = K(1 +
p
2L
i
)
2
W }K > 1
L
i
≥ 0
B B BB54 KrΘ
i
} v CB 5 w B J 44 44 } v B 5rB B B4B B KB KKB C BJ Kr wB KJ } CB4 B 5rB 5BBJB 44B X B B B 5B4w B B KJB } ~ 45 K 5B ¥ B 5rB B B 45 5α(n)
n
} 3 C 5KK B KJ Jα(n)
5BB 5n
Kr 5 J 44 B B 45B4 BK5 B W } } 5 B 5 B KJ 44 B 5Bn
5KK B W } } W } Cq B 5rB BB v r 44
α(n)
445B v KKB v W B4 qBJK 4K C C 4BB } |vlog n
3 5qY ~ X B5 4B 44 B 5 K4K }ϕ
log log n
EB 5 C 4BB 5 44 BB 55 BJwB } |vvlog n
BY BY | B4 B B B 5K ¥ 4 B5qBJ5 E ~} | Y 4 B45 C C 5q K }ϕ
β
n
β
log log n,
v B 44B BB0 < β < 1
55 BJwB } v 5B v B | B4 B KKB 4 BJB B KJB B KrB B45 CqBKKB } v 5B | XX | XW | B4 B KKB 4 BJB B KJB B KrB B rKB 4BB } J 5r C α(n) = 2
5 C ¥ 5 BJC B 5rB 5BBJB 44B B KJK KB J B 44 BBK 5B W } 44B 5K B 5 wB KJ }~ B B 54B 454KK 4B CB B 44B W } 5KK B 44BB 5 C } MNNT M N Q MMQO QOMN NQ C ¥ K B B 5 B5 CB 5r C } B C B KB 5rB CK B 4 } 4K5q B KKB BJ B 5q ¥ 5 44α(n)
B W } } B B B4f
θ
, θ
∈ Θ
4B B BB } BwB B 5 K4JΘ =
R
m
B KrBΘ
i
, i
∈
I
5K4 B B BBwB5B B KB|Θ
i
|
5KK 5CB 5B B BBwKrB 4B } } 3 BB 4K J B5 BBJ B B J BK5 B 4Bx
n
Bf
θ
Q
n
i=1
f
θ
(x
i
)
} 3 B B 5B 5BBJB § Y 44 B B 5Bθ
7→ f
θ
(x)
K5 CK Bqθ
?
4I(θ
?
)
r,s
=
−
θ
?
·
∂
2
log f
θ
(.)
∂θ
r
∂θ
s
|θ=θ
∗
¸
4 B5
h., .i
B CB5 B K B K BB54k.k
} 3 BB J C BB CBKb
θ
θ
∗
K 45n
54JB B4 J√
n(b
θ
− θ
∗
)
B B KJK K } B Bθ
∗
i
b
θ
i
B 5B qB B5Bθ
∗
b
θ
B Kr 4Θ
i
} B B B B KJB BB 6n
°
°
°b
θ
− θ
∗
°
°
°
2
∼ χ
2
(m)
n
°
°
°b
θ
i
− θ
i
∗
°
°
°
2
∼ χ
2
(
|Θ
i
|)
n
°
°
°(θ
∗
− θ
∗
i
)
−
³
b
θ
− b
θ
i
´°°
°
2
∼ χ
2
(m
− |Θ
i
|)
W } E B°
°
°θ
∗
− b
θ
i
°
°
°
2
=
kθ
∗
− θ
∗
i
k
2
+
°
°
°θ
∗
i
− b
θ
i
°
°
°
2
°
°
°b
θ
− b
θ
i
°
°
°
2
=
kθ
∗
− θ
∗
i
k
2
+
°
°
°(θ
∗
− θ
∗
i
)
−
³
b
θ
− b
θ
i
´°°
°
2
−2
D
θ
b
− θ
∗
, θ
∗
i
− θ
∗
E
.
W } V B J 5 B5 5 B X BBL
2
W } 3 5q 6·D
b
θ
− θ
∗
, θ
∗
i
− θ
∗
E
2
¸
≤
1
n
kθ
∗
i
− θ
∗
k
2
·
n
°
°
°b
θ
− θ
∗
°
°
°
2
¸
→
n→∞
0.
B J CK B K4J 5 4 } w K KB C45 ¥ B4BK(θ, θ
0
) =
−
Z
f
θ
(x) log
f
θ
0
(x)
f
θ
(x)
dx
≥ 0.
v KB K BΦ
w5B 4 44 ¥ K B | BY | B } B B BKK B B CK 5 K 4 K5 Cw K Bq } θ
Bθ
?
62K(θ
∗
, θ) =
kθ
∗
− θk
2
(1 + o(1)).
W } = ( 8 -H<:&*/ %< A-+/-& %( ? 8 +& ~ Kk.k
q44 K4J Bq4 CK 5 W } B K KB BJ 54 B45w Kr } i
∈ I
¥ 4 BJ 4B 5q ¥Θ
i
5KK B5 CBKθ
∗
B·°
°
°b
θ
i
− θ
∗
°
°
°
2
¸
.
W } ~ 4K5q 5BB B 5qB 44 5r W } }V = & A-+/-& ~ BJ W } B 55BB CB } B B K 5KK CBK 5r C } BB B 44B W } V B BJB 4 K K ¥ 5B B 55 ~
2
B X BK·
n
°
°
°b
θ
i
− θ
∗
°
°
°
2
¸
·
n
°
°
°θ
∗
i
− b
θ
i
°
°
°
2
¸
+
·
n
°
°
°b
θ
− b
θ
i
°
°
°
2
¸
−
·
n
°
°
°(θ
∗
− θ
∗
i
)
−
³
θ
b
− b
θ
i
´°°
°
2
¸
4B B B B KJB W } K K BCBK|Θ
i
|
m
− |Θ
i
|
} K B·
n
°
°
°b
θ
− b
θ
i
°
°
°
2
¸
BB W } BKBh
2nK(b
θ, b
θ
i
)
i
=
·
2n
b
θ
·
− log
f
θ
b
i
(.)
f
θ
b
(.)
¸¸
−2
n
X
j=1
log
f
θ
b
i
(x
j
)
f
θ
b
(x
j
)
.
W } Y B BK C45q BJ W } KKB B 6−
n
2
n
X
j=1
log
f
θ
b
i
(x
j
)
f
θ
b
(x
j
)
+
2
|Θ
i
| − m
n
~ KKBBBΘ
i
BBKB B 5 ¥ BB J C 4 B 5r v K v BB K 44 54 W } 6 v(k) =
−2
n
X
j=1
log f
b
θ
i
(x
j
) + 2
|Θ
i
|.
W } ~ C K 45B 5 5qBα(n) = 2
5 44 } Q MO Q NPNN } Bq B BYY 4B 44B B KJB B KrB 5qBB 5rB CK } ~ BJ B KrB 5K4 B K } BJS
?
4B Bθ
∗
K 4B B 6 902-:&b
S
b
S =
(
(S), S
⊂ [[1, m]]).
α(n)
α(n)
→ ∞
α(n) = o(n)
b
S
S
?
log log n = o(α(n))
α(n) = o(n)
b
S
S
?
v 4B K B CB C K 5rB 5q ¥ 44 C C B5 J K 55 J C Bq } ¥ K 5qB
α(n) = (log log log n)
α
, α > 0
B BB 55 4 BK B } ~ 5q ¥
α(n) = (log n)
γ
, γ > 0
J 55 BJwB } = & A-+/-&ϕ
β
EB 5 J w } B CB 5rϕ
β
44 B 40 < β < 1
α(n) = n
β
log log n.
W } X ~ 5q ¥ B Bβ
K4BB 44B BB q4rK W } } 55 BJwB B BK4 B B } K Bβ = 0
5rϕ
} v K4B CB B B B BB } n
¥ 4 BKβ
K 44 B B 5rB BB BJB B B W }} 5 B B BB B 5rϕ
β
B J B Bβ
} ~ B B 5rBβ
B 6β
= (log 2
− log log log n)/ log n
β
= (log log n
− log log log n)/ log n
W } B J
β
< 0
rB Jn
≥ 1619
} EB B B 5rϕ
β
B BB B B 5rBβ
BB 6β
=
log log n
log n
= 1
− β
W } W 5B CB 5rϕ
β
β
< β < β
} B 5BB J JC 5q ¥ 5qβ
B B KB 4BB B BK } B Jβ
< β
CB KB 5rϕ
β
4B B J 5r |v} STROQN MNONNT B5 CB B 5rB B B 4BBB B 5 K4K 5K ¥ 4 5 B } 5K ¥ 4 B B54 B45 KrB } EB 5B B KrB B 45B } K4q W } WJ B JB J BC45 5 6
b
S =
K(
v(S), S
⊂ [[1, m]])
W } 5K ¥ 42
m
} v 5K ¥ 4 ¥ B B 44 B ¥ 45 B K5q } B 45B 5 B K4qB CB B 5rB CK B K4q } k\haci gape`f`\] i v K4q B B4 Bq B BYY } ¥ B 5r BJ B B KrB B B 5KK 445 BKB BS
?
b
S
4 5KK B 6(
v=
v([[1, m]])
b
S =
©
j
∈ [[1, m]],
v≤
v([[1, m]]
\ {j})
ª
.
W } Bb
S
5 B 5KBB J 5r KB v≤
v([[1, m]]
\ {j})
} v K4q 45BB 55m + 1
5rB } k\haci gape`f`\] i ]b if_ki ~ K4q 5K B4 5 B5KK 445 5r BJ B B KrB B 4B BKS
?
(
v=
v(
∅)
b
S =
©
j
∈ [[1, m]],
v(
{j}) ≤
vª
.
W } V v B5 BB5KBB J5rKB } v K4q 45BB 4K 55m + 1
5rB } B B 5q J K4q 5K B4 B KB w B J 454 K4q } v K J B J4B v,
v(
{j}), j = 1, . . . , m,
B BJB B B BB CBK B 5 KB CKB J 5B B4B K4q W } } k\haci gape`f`\] i ci_gibc`b\i v K4q CB W } 4KB 5KBB4B B 5r } ~ 54 B 4 5 B K 4 C4B } 5KK5 ¥ BS
(0)
= [[1, m]]
v(0)
=
v(S
(0)
).
Kr 4 B B45B B 5B C4KC
(1)
=
n
j
∈ S
(0)
,
v¡
S
(0)
\ {j}
¢
≤
v(0)
o
.
Y W } W ~ B K4qB 5KB 5K ¥ 4 } 4q v K ¥ 4 ¦ W }
2
m
v K W }m + 1
v K B4 W } Vm + 1
v K B5 W } }≤ m(m + 1)/2
5B B 5KBJ
(1)
4 KB 5r 5BB 445 5q 4 6J
(1)
=
K¡
v¡
S
(0)
\ {j}
¢
, j
∈ C
(1)
¢
S
(1)
= S
(0)
\
©
J
(1)
ª
v(1)
=
v(S
(1)
).
~ C4k
≥ 1
44 54 5r C4k + 1
6C
(k+1)
=
n
j
∈ S
(k)
,
v¡
S
(k)
\ {j}
¢
≤
v(k)
o
J
(k+1)
=
K¡
v¡
S
(k)
\ {j}
¢
, j
∈ C
(k+1)
¢
S
(k+1)
= S
(k)
\
©
J
(k+1)
ª
v(k+1)
=
v(S
(k+1)
).
5 4k
f
+ 1
C
(k
f
+1)
=
∅
} v B J 5r JC C B 5KBB B BS
(k
f
)
} B B 5 54 5qBBBB 5 B 5KK BKS
?
} B J B B J4BC
(.)
, J
(.)
, S
(.)
,
v(.)
, k
f
B K4q 5K B5 B 4B } ~ 5K ¥ 4 5 K4q B 5 BB 4KB B 4m(m + 1)/2
} ape^i]\k_ ci_ pk\haci_ ~ W } W4BK B 5K ¥ 4B B K4qB 45B }~ B K4qB 5Kw B 4B B 5K ¥ 4 K K w KrB 5qBB B45 KrB }v BB J BB B rB K4K B K5q } E B B K B45w CK J B 5 B[[1, m]]
5 B5 Kr BB J K4q } MR 5B 5 5q B B 4B4 C4 44 B 5rB Cw K BB4 qBJ B B BB } B B 54 B KrB B5B 5B 5rB v W }ϕ
β
W } X B4B B B 5B4B B5B KKB BJB B4 }EB K4qB 5KB B K B W } } B B4B 5 CK J 5r 4 B5 5Kw ¥ 4 CB } 5B 5 K4KB B 5B B B K J 554 4 5 B K4qB 5BBJB B 5B KB 55 }
C B B 4B 4B