Plans Syst´ ematiques Quasi-´ equilibr´ es
Lionel Qualit´ e et Yves Till´ e
Universit´ e de Neuchˆ atel
novembre 2012
Plans ´ equilibr´ es
Les ´ echantillons s sont les vecteurs de R N dont les coefficients sont les indicatrices de s´ election des unit´ es : les sommets du cube [0, 1] N . Un plan p(·), avec probabilit´ es d’inclusion π = (π 1 , . . . , π N ) est
´
equilibr´ e sur X ssi
X b HT = X
k∈s
x k
π k = t X , ∀s t.q. p (s) > 0.
Il n’en existe que si un plan affine, dont la direction et la position sont
fonctions de X et π passe par des sommets du cube, et que π est une
combinaison convexe de ces sommets.
M´ ethode du Cube
Propos´ ee par Deville et Till´ e (2004) : marche al´ eatoire qui reste dans le plan affine des contraintes, et va se coller sur des faces du cube de dimensions de plus en plus basses.
Si les sommets de l’intersection entre le cube et l’espace des
contraintes ne sont pas tous des sommets du cube, il peut rester un probl` eme d’arrondi (mˆ eme si un plan ´ equilibr´ e existe).
Celui-ci concerne au maximum m unit´ es, o` u m est le nombre de variables auxiliaires dans X.
C’est en g´ en´ eral le cas lorsque l’on utilise des variables auxiliaires
“num´ eriques”.
Fonction de coˆ ut
Pour un plan parfaitement ´ equilibr´ e, var( X b HT ) = 0.
Lorsque l’on n’a pas de plan ´ equilibr´ e, on veut minimiser un crit` ere, par exemple
C (p) =
m
X
`=1
var( X b ` ) t X 2
`
.
C’est une fonction lin´ eaire en p.
Le probl` eme d’arrondi ` a la fin de la phase de vol de la m´ ethode du
cube est trait´ e de cette mani` ere.
Repr´ esentation des plans
On liste tous les ´ echantillons (evt. de taille fixe n) comme colonnes d’une matrice S.
Un plan de sondage avec probabilit´ es π est un vecteur p de nombres positifs ou nuls dont la somme vaut 1 et tel que Sp = π.
L’ensemble des plans de sondages C π est un polytope convexe de R 2 N (evt. R C
n N ).
Support d’un plan p(·) : Q p = {s t.q. p(s) > 0}.
Plans ` a support minimal
Les minima de C (·) sont atteints en des sommets de C π : les plans extremaux (et si on en est capable, on utilise alors l’algorithme du simplexe pour trouver un minimum global).
On sait caracteriser les plans extremaux : ce sont les plans ` a support minimal.
Support minimal (pour π) : ensemble d’´ echantillons Q tels que π ∈ Conv(Q) et R ( Q ⇒ π ∈ / Conv(R).
Remarque : le cardinal de Q ne d´ epasse pas N + 1 (evt. N), cf.
Wynn (1977), et ˆ etre un support minimal ne veut pas dire qu’il n’existe pas de support de cardinal plus petit.
On trouve un algorithme pour de tels plans dans Deville et Till´ e
(1998).
Plans syst´ ematiques
Le plan syst´ ematique est aussi un plan ` a support minimal (Pea et coll., 2007).
Faire les sommes : V 0 = 0, V 1 = π 1 ,. . . , V k = P k
i=1 π k , g´ en´ erer une r´ ealisation u de U ([0, 1]), et s´ electionner tous les k tels que
V k−1 < u ≤ V k .
Algorithme d´ epend de l’ordre de la population : l’ensemble des plans syst´ ematiques P SYST est l’ensemble des plans obtenus pour tous les ordres possibles.
On a dans Pea et coll. (2007) un algorithme rapide pour calculer le
support et le plan p(·).
Groupe sym´ etrique
P SYST est l’orbite du plan syst´ ematique p e (·) (e d´ esigne l’ordre initial) sous l’action du groupe de permutations Σ N .
Le N−cycle (1, . . . , N) et le retournement (1, . . . , N) 7→ (N, . . . , 1) ne modifient pas p e : ils sont dans le stabilisateur S e .
Sauf cas particuliers, ces deux permutations engendrent S e , qui est alors de cardinal 2N.
P SYST est en bijection (not´ ee Ψ) avec Σ N /S e , et a un cardinal en g´ en´ eral ´ egal ` a (N − 1)!/2.
Trop gros pour une recherche exhaustive, et trouver le minimum de
C (·).
Distance sur le groupe sym´ etrique
On cherche un minimum local : d´ epend d’une m´ etrique sur Σ N /S e . Si G est un ensemble sym´ etrique de g´ en´ erateurs de Σ N , on peut d´ efinir d G (σ, τ ), le nombre minimum d’´ el´ ements de G pour ´ ecrire στ −1 .
Graphe des points adjacents : graphe de Cayley. Si G est l’ensemble des transpositions, distance de Cayley.
Distance transport´ ee sur Σ N /S e par δ G (p 1 , p 2 ) = inf
σ∈Ψ −1 (p 1 ),τ ∈Ψ −1 (p 2 ) d G (σ, τ ).
Minimisation locale
Propri´ et´ e :
δ G (p 1 , p 2 ) = 1 ⇔ ∀σ ∈ Ψ −1 (p 1 ), ∃τ ∈ Ψ −1 (p 2 )|d G (σ, τ ) = 1 (1)
Algorithme :
1 On part d’un ordre initial (ex: selon π k croissants, voir Hartley, 1966),
2 On essaye les transpositions jusqu’` a en trouver une qui fait baisser C(·), que l’on applique,
3 Alternativement, on n’applique que la meilleure transposition,
4 On recommence jusqu’` a ne plus arriver ` a faire baisser C(·).
Grˆ ace ` a la propri´ et´ e 1, on arrive bien ` a un minimum local pour δ G .
En ne mettant dans G que les transpositions (i, i + 1) on acc´ el` ere les
calculs de C (·), mais on p´ ejore les r´ esultats.
Exemple 1 : faible dimension
N = 7, n = 3, m = 2, π = (.11, .51, .31, .61, .32, .41, .73) x 1 = (10.42, 8.31, 10.05, 8.55, 14.49, 9.46, 13.70),
x 2 = (7.28, 11.72, 16.23, 5.42, 14.27, 6.86, 13.18)
Il y a 29 0 402 plans extremaux, le coˆ ut minimum est 0.09385,
Il y a 360 plans syst´ ematiques, les minima locaux avec la distance de Cayley sont 0.09631, 0.101353, et 0.12685
Le coˆ ut du plan ` a entropie maximale est 0.21960.
Exemple 1 : suite
Distribution de C (·) sur les plans extremaux et les plans syst´ ematiques.
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
● ●●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●●
●
●
●●
●●
●●
extremaux systématiques
0.10.20.30.40.5
Minimum atteint selon ordre de d´ epart :
C (·) 0.09631 0.101353 0.12685
proportion 52.4% 32.2% 15.4%
Exemple 2 : MU284
π ∝ P75, auxiliaires : P75, RMT85, SS82, ME84, n = 20, N = 282 Cube : 10’000 ´ echantillons, Syst´ ematique : P75 tri´ e.
Plan ` a entropie maximale : C = 0.20497, Cube : C b = 0.00304,
Syst´ ematique : C < 0.0004 en utilisant toutes les transpositions, et
C = 0.00205 avec les transpositions (i, i + 1).
Exemple 2 : MU284 suite
Coˆ uts des ´ echantillons s´ electionn´ es par Cube, et dans le support du plan syst´ ematique trouv´ e.
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
Ech. Cube Ech. SYST
0.00 0.01 0.02 0.03 0.04 0.05
Exemple 3 : Utilisation comme phase d’atterrissage
N = 1000, n = 150, m = 30 (π, 1, et 28 normales ind´ ependantes), 1000 ´ echantillons tir´ es avec atterrissage par ´ elimination de variables, et 1000 autres par plans syst´ ematiques,
Coˆ ut moyen par ´ elimination : 0.00189, Coˆ ut moyen par syst´ ematique : 0.00434
Variables trait´ ees de mani` ere in´ egale avec la m´ ethode d’´ elimination,
mais jusqu’ici meilleurs r´ esultats qu’avec le syst´ ematique.
Exemple 3 : Atterrissage suite
CV 2 pour chaque variable, ´ elimination et syst´ ematique
●
●●
●
●
●
●●
●
●●
●●
●●
●●
●●
●●
●●
●●
●
●●
●
●
●●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●●
●
●●
●●●●●●
●
●●
●●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●●
●●
●●
●●
●
●●
●●●●●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●●
●●
●●
●●
●
●●
●
●
●●
●●
●●
●●
●●
●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●●●●●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●
●●
●●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●●
●●
●●
●
●
●●
●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●●
●●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●●
●
●●
●●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●●
●●
●●
●●
●
●●
●
●
●
●
●●
●●
●
●●●●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●
●●
●●
●
●●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●●●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●