telecharger

(1)

Université Pierre et Marie Curie – Paris VI

Mémoire de stage de Master 2 ATIAM

Estimation du tempo perceptif et

réduction des erreurs d’octave du tempo

Joachim FLOCON-CHOLET

Sous la direction de Geoffroy Peeters

1er mars 2012 – 31 juillet 2012

Institut de Recherche et Coordination Acoustique/Musique 4, place Igor Stravinsky 75004 Paris



(2)

(3)

(4)

x

y

T

_a

α

(5)

(6)

x X p(x)

E E x y

N (z | µ, Σ) µ

Σ

T

_e

T

_a

BP M

M F CC

GM M

(7)

(8)

(9)

(10)

(11)

M T T

T

_i

i = 1, ..., 5

M T

_i

M

T

^�

T

^�

= arg max

Ti

p(T

_i

| M)

p(T

_i

| M ) T

_i

M

(12)

y

T F (y) F M ACF (y)

T F (y) × F M ACF (y)

(13)

Au d io O n se t D e te ct io n F u n ct io n

Pe ri o d ici ty Est ima tio n T e mp o / Me te r Est ima ti o n Me te r/ Be a t Su b d ivi si o n Te mp la te

R e a ssi g n e d Sp e ct ra l En e rg y F lu x C o mb in e d D F T / F M- AC F V it e rb i d e co d in g o f MBST o ve r ti me

(14)

(15)

0 50 100 150

rock

country pop soul classic rock

rnb christian oldies

alternative disco

vocalists hip−hop

christmas dance reggae latin jazz hard rock

rap salsa funk folk Genres musicaux représentés dans la base de données

Nombre d’occurrences

(16)

| x ¯

_r

− x ¯

_s

| < 6% max { x ¯

_r

; ¯ x

_s

}

¯

x

_r

x ¯

_s

r s

¯ x

_r

= 1

n

_r

�

n i=1

x

_ri

6%

T a 4%

4%

T e

T a

(17)

T e = kT a k ∈ { 1/3; 1/2; 2/3; 2; 3 }

% T

_e

= 2T

_a

T

_e

= T

_a

/2 T

e

= 2/3T

a

T e = 3/2T

a

T

e

= 3T

a

T

_e

= T

_a

/3

± 6%

% % %

T

e

= 2T

a

T

e

= T

a

/2 T

e

= 3T

a

T

_e

= T

_a

/3

%

(18)

!

"#"

"#"$

"#"$%

"#&"$%

"#%&"$

"#"

"#%"

(19)

t

S(f

_k

, t) f

_k

f

_k

t

i

l ∈ [0, 11] l

n(f

_k

, t

_i

) = 12 log

₂

� f

_k

f

_ref

�

mod 12

(20)

f

_ref

t

_i

c(l, t

i

) = �

fk n(fk,ti)=l

| S(f

_k

, t

i

) |

²

l = 0, 1, ..., 11

C(l, t)

t t

C(l, t) t

_i

L t

_i

R t

_i

L R

T

_h

T

_h

∈ [30, 200]

L = [t

i

− αT

_h

, t

i

] R = [t

i

, t

i

+ αT

_h

] T

_h

α = 4

t

_i

µ

_L

C(l, L) µ

_R

C (l, R) µ

_L

µ

_R

d(L, R) = 1 − µ

_L

· µ

_R

|| µ

_L

|| · || µ

_R

||

C

_i

(ω, T

_h

) C

_i

i ω

T

_h

(21)

Temps [sec]

Tempo assumption

5 10 15 20 25

40 60 80 100 120 140 160 180

Freq [bpm]

Tempo assumption

0 20 40 60 80 100 120 140 160 180 200

40 60 80 100 120 140 160 180

T

_h

T

_h

(22)

t

_i

T

_h

∈ [30; 200]

t

_i

L kmax

r(t

_i

) =

�

ti+L/2 t=ti−L/2

�

N/2

k=kmax

| S(ω

_k

, t) |

²

�

ti+L/2 t=ti−L/2

�

kmax

k=1

| S(ω

_k

, t) |

²

N L T

_h

/2 kmax

t

_i

r T

_h

r

_j^�

(t

_i

) = r(t

_i

− (j − 1)T

_h

)

j ∈ { 1; 2; 3; 4 } r

^�_j

(t

_i

) �

j

r

^�_j

(t

_i

) = 1 r

_j^��

(t

_i

) = 1 − r

^�_j

(t

_i

)

r

_tot

(t

_i

) = r

^��_j=1;3

(t

_i

) − r

_j=2;4^��

(t

_i

)

T

h

t

i

r

_tot

(t

_i

) r

_tot

(t

_i

+ T

_h

)

B(t

_i

, T

_h

) r

_tot

(t

_i

)

r

_tot

T

_h

B(ω, T

_h

) w

B(t

_i

, T

_h

) B(w, T

_h

)

B

_i

(ω, T

_h

) B

_i

i ω

T

_h

(23)

Time [sec]

Hypothèse de tempo (Th)

5 10 15 20 25

40 60 80 100 120 140 160 180

Freq [bpm]

Hypothèse de tempo (Th)

0 20 40 60 80 100 120 140 160 180 200

40 60 80 100 120 140 160 180

r T

_h

r T

_h

t

_i

S(t

_i

, t

_j

) t

i

t

j

v

_t_i

t

_i

S(t

_i

, t

_j

) = d(v

_t_i

, v

_t_j

)

(24)

S(t

_i

, t

_j

) L(t

_i

, l

_j

) l

_j

= t

_j

− t

_i

l

j

l

j

t

_i

Matrice d’auto−similarité

Temps [sec]

0 2 4 6 8 10 12 14 16 18 20

0 5 10 15 20

Matrice de retard

Temps [sec]

Retard [sec]

0 2 4 6 8 10 12 14 16 18 20

0 5 10 15 20

0 20 40 60 80 100 120 140 160 180 200

0 0.5 1 1.5

2 x 10

⁻³

Fréquence de répétition [BPM]

(25)

E(i) = �

f

| X(f, t

i

) | − | X(f, t

i−1

) |

X(f, t

_i

) x t

_i

E(i)

(26)

0 5 10 15 20 25 30 0

2 4 6 8

Temps [sec]

Amplitude

Fonction d’energie

0 50 100 150 200

0 0.05 0.1 0.15 0.2 0.25

Spectre de la fonction d’energie

Freq [BPM]

Amplitude

(27)

(28)

z z D K

p(z) =

�

K k=1

π

_k

N (z | µ

_k

, Σ

_k

)

N (z | µ

_k

, Σ

_k

) N (z | µ

_k

, Σ

_k

)

N (z | µ

_k

, Σ

_k

) = 1

(2π)

^D/2

| Σ |

^1/2

exp

�

− 1

2 (z − µ

_k

)

^T

Σ

⁻¹

(z − µ

_k

)

�

µ

_k

D Σ

_k

D × D

| Σ | Σ

π

_k

π

_k

z

k

^eme

�

K k=1

π

k

= 1, π

k

� 0

x = [x

₁

x

₂

...x

_N

] Y = [y

₁

y

₂

...y

_N

] z

x

_i

y

_i

z =

� y

_i

x

_i

�

π

_k

µ

_k

Σ

_k

p(z)

p(x

_i

, y

_i

) EM

x y

F (y) = E [x | y] =

�

K k=1

h

_k

�

µ

^x_k

+ Σ

^xy_k

(Σ

^yy_k

)

⁻¹

(y − µ

^y_k

) �

h

_k

(y) = π

_k

N (y | µ

^y_k

, Σ

^yy_k

)

�

K

k=1

π

_k

N (y | µ

^y_k

, Σ

^yy_k

)

y k

^eme

Σ

_k

=

� Σ

^yy_k

Σ

^yx_k

Σ

^xy_k

Σ

^xx_k

�

µ

_k

=

� µ

^y_k

µ

^x_k

�

x

y

(29)

x

m

i

m

_i

=

� T a

_i

T e

_i

z = [y x]

^T

x

y

(30)

i C

_i

B

_i

s

_i

e

_i

C

_i

B

_i

s

_i

e

_i

T e

_i

C

_i

B

_i

s

_i

e

_i

k =

� 1 4 ; 1

3 ; 1 2 ; 2

3 ; 3

4 ; 1; 1.25; 1.33; ...; 2

�

C

i

B

i

T h

k D = 12

D = 48

T

_{T e}^T

k

!" # $ %&'(

T e T e = 106.3

c

^�_i

b

^�_i

s

^�_i

e

^�_i

(31)

y

_i

=



 

 c

^�_i

b

^�_i

s

^�i

e

^�_i



 



z

_i

=

� y

_i

x

i

�

=



 

 c

^�_i

b

^�_i

s

^�_i

e

^�_i

T a

_i



 



T e

T T e

= 1

T e

T e = T a T e = 2T a

EM

(32)

loglikelihood

_N

N loglikelihood

_N

loglikelihood

_N₋₁

− 1 < 1

⁻¹⁰

(33)

T a

T e

T a T a T e

T

a

T

e

Obs T

_a

K

6%

T a

K

K = 20

K = 4 K = 8 K = 12 K = 16 K = 20 K = 24

K

K = 20

(34)

σ

σ = 1.45 − 4.1 σ = 2.96 +0.9 σ = 3.50 +0.4 σ = 2.31 − 1.13 σ = 1.91 − 0.36 σ = 2.33 − 2.05

T a

1%

α

αT e = T a

T e

^�

= ˆ αT e α ˆ

α α =

_{T a}^{T e}

T e α

α α

T e = 2T a T e =

^{T a}₂

α =

¹₂

α = 2

(35)

α = 1 α ∈ �

₁

2

; 1; 2 �

α α ∈ �

1/2; 1; 2 � α ∈ �

1/2; 1; 2 �

± 0.2 α = 0.65

α = 0.5

T a T a ∈ [30, 200]

K

K = 2 K = 4 K = 8

% σ % σ % σ

1.79 2.36 2.22

3.27 3.39 2.38

1.79 3.69 1.94

2.24 3.78 3.26

2.16 2.29 3.00

2.34 2.03 2.65

2.33 3.53 2.95

3.11 1.72 3.16

2.52 2.51 2.63

2.31 1.99 2.82

1.43 2.01 2.01

2.59 2.86 2.84

α

α = 1 α = 1/2

α = 2

(36)

α = 1 T e = 2T a α = 1/2 T e = T a/2 α = 2

p(y) =

�

K k=1

π

_k

N (y | µ

_k

, Σ

_k

)

y K

µ

_k

Σ

_k

(37)

K = 2 K = 4

% %

σ = 3.22 σ = 3.08

σ = 4.03 σ = 2.71

σ = 2.53 σ = 2.29

σ = 2.97 σ = 3.03

σ = 3.70 σ = 3.90

σ = 3.08 σ = 2.23

σ = 2.83 σ = 2.46

σ = 3.04 σ = 2.39

σ = 2.72 σ = 1.63

σ = 3.56 σ = 2.30

σ = 2.28 σ = 2.90

σ = 3.82 σ = 2.76

T a

(38)

T e = 2T a T e = T a/2

α αT e = T a

10%

(39)

(40)

N (z | µ

_k

, Σ

_k

) = 1

(2π)

^D/2

| Σ |

^1/2

exp

�

− 1

2 (z − µ

_k

)

^T

Σ

⁻¹

(z − µ

_k

)

�

µ

_k

D Σ

_k

D × D

| Σ | Σ

x D N (x | µ, Σ)

x x

_a

x

_b

x

_a

M x x

_b

M − D

x =

� x

_a

x

_b

�

µ µ =

� µ

_a

µ

_b

�

Σ Σ =

� Σ

aa

Σ

_ab

Σ

_ba

Σ

_bb

�

Σ

^T

= Σ Σ

aa

Σ

_bb

Σ

_ba

= Σ

^T_ab

Λ

Λ ≡ Σ

⁻¹

(41)

Λ =

� Λ

aa

Λ

ab

Λ

_ba

Λ

_bb

�

Λ Σ

Λ

_aa

Σ

_aa

−

¹₂

(x − µ)

^T

Σ

⁻¹

(x − µ) = −

¹₂

(x

_a

− µ

_a

)

^T

Λ

_aa

(x

_a

− µ

_a

) −

¹₂

(x

_a

− µ

_a

)

^T

Λ

_ab

(x

_b

− µ

_b

)

−

¹₂

(x

_b

− µ

_b

)

^T

Λ

_ba

(x

_a

− µ

_a

) −

¹₂

(x

_b

− µ

_b

)

^T

Λ

_bb

(x

_b

− µ

_b

) N (x | µ)

− 1

2 (x − µ)

^T

Σ

⁻¹

(x − µ) = − 1

2 x

^T

Σ

⁻¹

x + x

^T

Σ

⁻¹

µ + const x µ

^T

Σ

⁻¹

µ

Σ

⁻¹

x

x Σ

⁻¹

µ

p(x

_a

| x

_b

)

µ

_a_|_b

Σ

_a|b

x

_a

x

_b

x

_a

− 1

2 x

^T_a

Λ

_aa

x

_a

p(x

a

| x

_b

) Σ

_a_|_b

= Λ

⁻_aa¹

x

_a

x

^T_a

�

Λ

_aa

µ

_a

− Λ

_ab

(x

_b

− µ

_b

�

Λ

^T_ba

= Λ

_ab

x

_a

Σ

⁻_a|b¹

µ

_a|b

µ

_a|b

= Σ

_a_|_b

�

Λ

_aa

µ

_a

− Λ

_ab

(x

_b

− µ

_b

) �

= µ

_a

− Λ

⁻_aa¹

Λ

_ab

(x

_b

− µ

_b

)

µ

_a_|_b

Σ

_a|b

Λ

� A B C D

�

−1

=

� M − M BD

⁻¹

− D

⁻¹

CM D

⁻¹

+ D

⁻¹

CM BD

⁻¹

�

M = (A − BD

⁻¹

C )

⁻¹

.

(42)

M

⁻¹

� Σ

_aa

Σ

_ab

Σ

_ba

Σ

_bb

�

−1

=

� Λ

_aa

Λ

_ab

Λ

_ba

Λ

_bb

�

Λ

_aa

= (Σ

_aa

− Σ

_ab

Σ

⁻¹_bb

Σ

_ba

)

⁻¹

Λ

_ab

= − (Σ

_aa

− Σ

_ab

Σ

⁻_bb¹

Σ

_ba

)

⁻¹

Σ

_ab

Σ

⁻_bb¹

p(x

_a

| x

_b

)

µ

_a_|_b

= µ

_a

+ Σ

_ab

Σ

⁻_bb¹

(x

_b

− µ

_b

) Σ

_a|b

= Σ

_aa

− Σ

_ab

Σ

⁻¹_bb

Σ

_ba

µ

_a_|_b

F (y) = E [x | y] =

�

K k=1

h

_k

�

µ

^x_k

+ Σ

^xy_k

(Σ

^yy_k

)

⁻¹

(y − µ

^y_k

) �

h

_k

(y) = π

_k

N(y | µ

^y_k

, Σ

^yy_k

)

�

K

k=1

π

_k

N (y | µ

^y_k

, Σ

^yy_k

)

F (y) = E [x | y] = µ

_x_|_y

h

_k

(y)

µ

_x_|_y

x

i

C

_i

(43)

!""# $%"&$

(44)

!""# $#!$

!""# $%$

(45)

(46)