HAL Id: hal-00367577
https://hal.archives-ouvertes.fr/hal-00367577
Submitted on 11 Mar 2009
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Codage arithmétique pour la description d’une distribution
Guilhem Coq, Olivier Alata, Christian Olivier, Marc Arnaudon
To cite this version:
Guilhem Coq, Olivier Alata, Christian Olivier, Marc Arnaudon. Codage arithmétique pour la descrip-
tion d’une distribution. TAIMA, May 2007, Hammamet, Tunisie. pp.65-71. �hal-00367577�
1
!" #$%$2
"&%$' !"2
% ($") #"'$*'1
1
+,-./,0.1/2 32 4,0567,01892:20;<<=1>,01.?:@A4B CDBE FGHF IJ KGLMNO HFNFP Q909/.:>.<2 C5,::2?291= C232R Q /,?>2
S
6= TGU VNVN FH NM Q,R TGUVNVN FNGL
WXYZ[\][^_X]`a[bcd^]efg hXebei\jd k
\
2
+,-./,0.1/2 E1l?,= m7,l2 20C.779?1>,01.?IJ K
GLMNO HFNFP Q909/.:>.<2 C5,::2?291= C232R Q /,?>2
S
6=nTGU VNVN FUFMQ,R TGUVNVN FUMG
[o[b[ZXoefei\`jeWdjhpaed^]efg hXebei\jd k
\
qrstur J
,/0,?0 39 >.3,l2 ,/1057601892</631>01v ,3,<0,01v 20901=1:,?0=2</1?>1<2 39 41O
?1797 w2:>/1<01.? +2?l05@?.9: ,//1x.?:y 9? .901= 2z>,>2 <.9/=, :6=2>01.? 32 7.3{=2: T
=2 >/10{/2 3|1?v./7,01.? BmCn D.9: </6:2?0.?: 2?:9102 9?2 2R02?:1.? 32 >2: 02>5?1892: 32
>.3,l2 y =|2:017,01.? ?.?O<,/,760/1892 3|9?2 31:0/1-901.? 20 =|1==9:0/.?: :9/=|51:0.l/,772
32:?1x2,9R 32 l/1:3|9?2 17,l2n
}~srs
C/10{/2:3|1?v./7,01.?@4w+@:6=2>01.?327.3{=2:@2:017,01.??.?O<,/,760/1892@
51:0.l/,772:n
¡¢£ ¤¥¦
§
£ £££
§
¨ © ª¢ £ «¬®¯°°°
± £ ²£ ³ £¥ ´ ¯
§
§
§
£ £ ¤ £ µ¶·µ
£ ¸ £ £ ²£¥ ¹ £ º £
££² ¤³ ¤ £ »£
¼¼°¢ £ ¤£ ½» ² ¤£ ¤
² £ ½ ¯¾¢¥ ´ ¾
£ £ £² ¤
½ ¥ ££ ³ © £ ¿¢
£ £ À¢ £ £ ¤ ¥
Á ÂÃÄÅ Å ÆÇÅ Å Ã ÈÉÊ ÇÅ ÆÊ Ë ÃÃÆ Ã Ë
ÌÍÎ ÏÐÑÒÓÔ ÔÕÖ×ÐØÙÚÛÔ
¹
E
½£ m
¨½£¥¦ ½ E
£ ܧ
C : E → ∪ i∈ N ∗ {0, 1} i
¥ £ C(x)
L(x)
¥ ¹L
§Ý £¤£ à³
§
»£ ᢠ¤££ £ £ ¤ ³
£ Ý» ½£ ¥ ¬
P
½½£ E
L = d− log P e
±log
£ £ © ½ ¯L
§Ý £ ££ ¤ ³
§
P
¥¶P (x)
L(x)
³½£¥£ £¤£
§
» « â
P
Q
» ½½£ E
IE P
£¤
P
âH(P ) := IE P [− log P ] ≤ IE P [− log Q] =: H(P, Q)
ã¼ä¹
§
P
£¤½Ü³ § Q
£¤
H(P, Q)
H(P )
¥¶ º £ åº £¥ µ £ ££ £
£ ¨½£ £¥
ÌÍÌ ÏæÒçÕÔè ÑÔ éÒ×êÐë ìÛíÖÙØíÔè
µî ïð
§

ã
µïïä ££ ¥
¦
(X n ) n∈ N ∗
©
§
£
E
µïï ¤k ∈ N
k
£ £
§
Ý £¤£
P (X n |X n−1 , . . . , X 0 ) = P (X n |X n−1 , . . . , X n−k )
n
¥´ £ Ü £ ± £ ££
n
ñ £î £ ²¥ ¦ µïï ¤ °
§
½£ £
¥
¬ £
k
² §½£ ¤ µïï ¤k
½ò ³
E
¥´i ∈ E
j ∈ E k
θ(i|j)
£½½£
§
i
²j
¥ (m − 1)m k
£θ(i|j)
j ∈ E k i
ò
m − 1
E
¸ © £¤§£ X
¥ ¬θ
£ ² x n = x 1 , . . . , x n
î ¤£
E
£ §½£ x n
£§ ©θ
¤ â
P(x n |θ) = 1 m k
Y
j∈E k
Y
i∈E
θ(i|j) n(i|j)
ã¯ä
§
n(i|j)
£ ½ ¤ i
²j
x n
¥ÌÍó ÏÐÑÒÓÔ Ò×ÙÖæìôÖÙÚÛÔ Ø×ôÑÙõÖÙö ÒÑÒØÖÒÖÙö
¹ £¤
§
££
I c = [0, 1[
¥¹ © x n ∈ E n
© £¤k
£½£ ² ¥ ¬
t ≥ 0
x t = x 1 , . . . , x t
£t
¨½£
t ≥ 0
ñt = 0
Ý £ ¤ ¥ ¬ £
(t + 1)−
² £ £ ½½£ âθ ˆ (t) (i|j) = n (t) (i|j) + 1
n (t) (j) + m
C.3,l2 ,/1057601892<.9/=,32:>/1<01.? 3|9?2 31:0/1-901.?
±
i ∈ E
j ∈ E k
n (t) (i|j)
n (t) (j)
£ ½ ¤ ³ i
²j
j
x t
ñn (t) (j)
§ j
©£ Ý î¥÷
j = x t−k+1 , . . . , x t
£¤ £ I c
£ £ ½½£θ ˆ (t) (i|j)
§
££ ©
i
E
¥÷ §£I c
£
©
x t+1
¥¦³ £ ¨½£
I c = [a, b[
£» » ½ ¨ £
d− log(b− a)e
³ I c
¥
÷
x n
£ ³ò £ ½¥ ¬ ££
E = {a, b}
£ ½£ ¼£ î
abaa
© £¤k = 1
¥¬d− log(7/24 − 1/4)e = 5
£ °¼°°¼
§
°¼°°°
1/4 ≤ 2 −2 + 2 −5 < 7/24
1/4 ≤ 2 −2 < 7/24
¥øùúû üý
C.3,l2 ,/1057601892</631>01v ,3,<0,01v 32 =, >5,þ?2
abaa
y=|./3/2k = 1 t x t I c θ ˆ ( t ) (.|.)
w6>.9<,l2G
∅ [0, 1) (a|a) = 1/2
(a|b) = 1/2 [0, 1 2 , 1)
L
a [0, 1 2 ) (a|a) = 1/2
(a|b) = 1/2 [0, 1 4 , 1 2 )
P
ab [ 1 4 , 1 2 ) (a|a) = 1/3
(a|b) = 1/2 [ 1 4 , 3 8 , 1 2 )
K
aba [ 1 4 , 3 8 ) (a|a) = 1/3
(a|b) = 2/3 [ 1 4 , 24 7 , 3 8 )
V
abaa [ 1 4 , 24 7 ) . . . . . .
nnn
ÿ
ÔìÒ×ÚÛÔ
£ £ £ ¤
k
£ î© £
§
¥ ¬ £ £ î £² ©
£ £
§
££ £ £ £ ³½£¥
ÊÅ Å ÉÅ ÆÃ ÉÉ ÅÆ ÅÄ È
µ £ ½£² £ ²£
§
â î
x n
£ £¤
k ˆ
¤ µïï x n
£¥¬
k ∈ N
Θ k
£ ²£ µïï ¤k
Θ
£ Θ k
¥ ½ £½ ¤²
θ ∈ Θ k
|Θ k | = (m −1)m k
¥¶££»
x n
£§ ²£Θ k
££
x n
© £¤k
C k (x n )
¥ ¹§ £ x n
£ ¤ µïï ¤
k ?
£k ?
C k (x n )
¥ ï
k ˆ
£¤ C k (x n )
IE [C k (x n )]
¥óÍÎ èÖÙìÒÖÙÐÕ ÑÔ íÒ õÐìØíÔÙÖô èÖÐõæÒèÖÙÚÛÔ
££
C k (x n )
£» º ¢ ££ IE [C k (x n )]
£ £ £ â k ∈ N
ε > 0
òθ k ∈ Θ k
n
â
nH (θ k ) + (1 − ε) |Θ k |
2 log n ≤ IE θ k [C k (x n )] ≤ nH (θ k ) + |Θ k |
2 log n + o(log n).
ãÀä·£ £¤£ ¸ £¤£
¤³¹
ã
¼ä â
§
θ k
£ ³© £¤
k
¨ ½ ½ ¨½£IE θ k [C k (x n )]/n
£ £§
H(θ k ) + |Θ k | log n/2n
¥ |Θ k | log n/2n
½ £¤ ³ £¤
H(θ k )
¥¶
x n
H(θ k )
−1/n log P (x n | θ ˆ k )
±θ ˆ k
£¤
»
§
½£
θ k
²£
Θ k
¥ £
ã Àä
² ¤
IE θ k [C k (x n )]
â·µ
(x n , k) = − log P (x n | θ ˆ k ) + |Θ k |
2 log n,
ã¾ä £ ï £ ½£² £ ²£ £ »
ˆ k =
¶{
·µ(x n , k) | k ∈ N }
¥µ ² ·µ
ã
·³ µä £ ³ ·µ
ã ¨ò
·³ µä ¹ ¼°¢ £ µïï
£¥ ¼¼¢¥
÷ ò³ ££ £
£ ª¢¥¶
§
£î
x n
© £¤k
§£ ²θ ˆ k
d− log P(x n | θ ˆ k )e
½¥¶ £ £ § £© » £
§
½£¥ ² ¤³ ¤ £ ³
²
§
£ £
|Θ k |
2 log n
£ £ ½ ²£¥ µ£ ££ £
§
¥
óÍÌ ÏÐìØÒ×ÒÙèÐÕ ÑÔè õÐÑÒÓÔè ÔÖ õ×ÙÖ×Ôè èÛ× èÙìÛíÒÖÙÐÕ ÑÛÕÔ Ïéé
´ £
x n
n = 2000
¤ µïï ¤k ? = 5
© ¯ ¥¤ ²
θ 5
£ H(θ 5 ) = 0.527
¥¹ î k = 0, . . . , 7
º £ £ © £¤
k
£ ³©£¤
k
£££» §½£ ï(x n , k) = − log P (x n | θ ˆ k )
£££ ·µ
(x n , k)
¥ £ § n
Ý ¼¥ ½ ³ ·µ £¤
k ?
ñ £ ¤»£ £ ¯¥À¥ ¤½ ¤§ £ ÜÝ £ » ½ £ ²
½
§
⣠² ï ³² ¡¥
ûüý
C.7<,/,1:.? 32:316/2?02:=.?l929/:32 >.3,l2:20 39 >/10{/2 609316n
ÆÆÃ Ã ÅÆ Å ÆÃ È ÄÃÉÉÅ
´ £ ³ ò
£¤ ¥ ¹
f
I
x n
££ ½ Π
I
©m
§££(I j ) j∈[[1,m]]
¥ÍÎ !Ô õ×ÙÖ×Ô Ø×ÐØÐèô
·£ ¤
x n
º © £¤ Π
¥
¬
i ∈ [[1, n]]
y i ∈ E = [[1, m]]
£ £¤§££ ££ ½x i
¥¬ x i
£ ³ y n
©£¤k = 0
£££¥´
L(y n |Π)
£ £ ¥¶i
Ý» § § £x i
©
y i
º © £¤ £¤
§
££
I y i
© £
Ý»
log l y i /r
±l y i
££
I y i
r
£ £¥£
§
x n
© y n
£L(x n |y n ) := P m
j=1 n j log l j − n log r
±
n j
£ ½
x i
½
I j
¥
£
x n
L(x n |Π) := L(y n |Π) +L(x n |y n )
¥·£³ £ £
ñ
£ £ © ³½£
§
L(x n |Π)
£¤¥ L(y n |Π)
·µ(y n , 0)
ã¾ä Ý £
§
² â
µ
(x n |Π) =
·µ(y n , 0) + L(x n |y n ) = − log P (y n | θ ˆ 0 ) + m − 1
2 log n + X m j=1
n j log l j −n log r
±
θ ˆ 0 (j) = n j /n
» §½£¥ ¦£ ã¯ä £ §µ
(x n |Π) = − X m j=1
n j log n j nl j
+ m − 1
2 log n − n log r
ãáä £ £ ² £ »£ À¢ £
£ ¤ ¥
ï ££ ²¥
½
I
£§ ©£ £ ò¤
Π
» ©R
§££ ¥´ £ £ ¨ £¥ ¿¢
§
£ò £
Π
» cR 2
£±c
¥ÍÌ ÔìØíÔè
¬ £
e −|x| /2
ã »£ ½ ¸ ¶µ ½£ µ"
8 × 8
«¬®ä §I = [−5, 5]
Π max
£ £²
2.10 −2
½ £ Ý ¯¥ãäñ£ ½ £ ¥ ¹ £¤ ¯áª
§
» £¤
§
I = [0, 255]
£ ¯¥ã½ä À¿ §££¥ £ » £ ² £ ¤
§
££ » ± £ ³
§
¥
£¤ £ À¿
§
» Ý À¥
ã
½ä¥££
¬¹´ Àᯠ© £¤ £
§
££ ² ½£¥
#
,$
#
-$
#
>$
û%ý&
1:0.l/,772>5.1:1 :9/ 9?2 +,<=,>12??2
#
,$2051:0.l/,772>5.1:1:9/=|17,l2 +6?,2?
K
N >=,::2:
#
>$y<,/01/ 32 =,31:0/1-901.? 1?101,=2 2? PUF>=,::2:
#
-$n
Íó ÏÐÕõíÛèÙÐÕ
´
§
© ï ¤ ½
¥ ¤½ ¤ £ ©
ò »£ » ³¥ ££¤£
² ¤ £ î ¤
¤
§
¥
#
,$
#
-$
û'ý
+|17,l2 ./1l1?,=2
#
,$20=|17,l2 /2>.?:0/9102
#
-$:9/
K
N?1x2,9R <.9/9? J
EDB 32 K
H@ UP 3 I
n
(
ÊËÊÅÅ
Ln
&
n ;),1)2n ; D2* +..) ,0 052 E0,01:01>,= 4.32= m32?01+>,01.?n ,--- ./0120345612 61 784690453
:
614/6;@LN T MLF<MP K
@LNMVn
Pn ß=1x12/ ;=,0, ,?3 C5/1:01,? ß=1x12/n C5.1>2 .v , PO3 >,9:,= ,90./2l/2::1x2 02R09/2 7.32= 9:1?l1?v./O
7,01.? >/102/1,n =044>/1 ?> 36@154561 A>44>/2@PV
#
NOLG$TLLNL<LPGL@PGG K
n
K
n +9>12?
I
1/l6 ,?3 Bx2: B.C2?5.=>n
&
.* 7,?D -1?: :5.9=3 -2 <90 1? , /2l9=,/ 51:0.l/,7n -E7,F
=/6G0GHE404H@LG T PV<VU
#
2=2>0/.?1>$@PGGFn
Vn ;-32=,C1C I=4,0.9,0@C5/1:01,? ß=1x12/@,?3 Q /636/1>J.9C2=n C5.1R 39 ?.7-/2 32 >.7<.:,?02:3|9?
7.3{=232 76=,?l2l,9::12? <,/>/10{/2:3|1?v./7,01.?:n KLM9> 361@/M2 ?N,7O=0/52@PGGGn
Un J
202/ wn Þ/9?*,=3@ m? J,2 4D9?l@ ,?3 4,/) ;n J
100n 7PQ013>2 51 F515989 R>23/5S4561 A>1@4T U
.T> 6/V 01P7SS;53045612 W X
>8/0; ,1 Y
6/904561 =/63>2251@Zn S
52 4m S J
/2::@PGGUn
Fn J
,9=Þn
&
.*,/3 ,?3 J22/D E>.00[1002/n ;/1057201> >.31?lv./ 3,0,>.7</2::1.?n S
2>5?1>,= B2<./0
S
2>5?1>,=/2<./0wA\I<
S
B<LNNV<GN@ LNNVn
Mn J./7, B1::,?2?n Þ2?2/,=1C23 \/,v0 1?289,=10D ,?3 ,/1057201> >.31?ln ,]F ^68/10; 6 Y
?>2> 0/3T 01P
R>Q>;6S9>14@PG
#K
$TLNH<PG K
@ LNMFn
Hn J./7, B1::,?2?n E0.>5,:01> >.7<=2R10D ,?3 7.32=1?ln 711HE404524H@LV
#K
$ TLGHG<LLGG@ LNHFn
Nn J./7, B1::,?2?@ S
2//D J
n E<223@ ,?3 I
1? B9n w2?:10D 2:017,01.? -D :0.>5,:01> >.7<=2R10Dn ,---
./0120345612 61 ,1 Y
6/904561 .T> 6/V@
K
H
#
P$T K
LU<
K
P K
@ LNNPn
LGn Þ132.? E>5*,/Cn I:017,01?l0523172?:1.? .v , 7.32=n
711HE404524H@
F
#
P$ TVFL <VFV@ LNMHn
LLn +nCn_5,.@ CnCnBnw./2,@,?3 CnBnÞ.?>,=x2:n ß? 3202/71?,01.? .v 052 ./32/.v , 4,/).x >5,1?n
E404524530;,1 Y
>/>13>
Y
6/E463T02453 =/63>22>2@V
#K
$T PM K
<PHP@PGGLn