D.1 Problem introduction
Precise automatic music transcription requires accurate modeling and identification of the spectral content of the audio signal. Whereas a deterministic model in terms of modulated periodic signals allows to distinguish different notes, the presence of multiple notes separated by octaves poses a big problem since they share the same periodicity, and hence completely overlapping spectral content.
Here we depart from the theory of the method based on cyclic correlation analysis, extending it by using the even and odd part of the periodic signature of the signal. In section D.2.3 we apply the method as a pitch determination algorithm on both synthetic and acoustic signal. Then, in section D.2.3.2 we use it for solving the octave ambiguity problem and compare it to a more sophisticated spectal method and, finally we conclude.
D.1.1 Illustration
The Octave of a note has a fondamental frequency which is twice the note,foctave = 2fnote. If the instrument is perfect, purely harmonic, the periodicity (temporal, Figure D.1) or the harmonics (spectral, Figure D.2) are completly shared. The contribution of the octave is shown in Figure D.3 for a guitar. In this condition, traditional pitch estimator failed to find the presence of the two notes, and an additional analysis must be used.
As for the interpretation effects detection, the state of the art is very limited and the Octave detection is, in the most case, ignored in automatic music transcription. In [169] an octave detector is proposed, the system use Support Vector Machine (SVM) and analyse the timbre of the mixed notes. Here we present a method which is equivalent of theOdd to Even Harmonics Energy Ratio (certainly used on several systems) [163] but the analyse is not completly performed in the spectral domain, we analyse the Odd and Even harmonics ratio in the time domain [170].
132 APPENDIX D. PERIODIC SIGNAL MODELING
Figure D.1: Illustration of the Octave Problem, Temporal point of view
Figure D.2: Illustration of the Octave Problem, Spectral point of view
D.2. PERIODIC SIGNAL MODELING 133
100 200 300 400 500 600 700
−20
−15
−10
−5 0 5 10 15 20
Frequence (Hz)
dB
Octave contribution
Note+Octave Note
Figure D.3: Difference between note and note+octave
D.2 Periodic signal Modeling
Musical signals are often modeled as the sum of sinusoids with time varying parameters and an additional noise. For an instrumental or a speech signal, the signal is also harmonic with fundamental frequency equal tof0.
s(t) = x(t) +n(t), (D.1)
x(t) =
NX−1 n=0
An(t) cos(2πfn(t)
fs +φn(t)) (D.2)
fn(t) = n f0(t) (D.3)
As defined in [143] the periodic signal can be expressed by itsgeneralized ACF, which is cyclic.
rPk =rk∗ δk,o,P, δk,n,P =
+∞X
i=−∞
δk,n+iP (D.4)
where ∗ denotes the convolution operator; and δ the Kroenecker delta. Its spectral expression is given by:
SP(f) =S(f) 1 P δ1
P(f), δf0(f) =
+∞X
k=−∞
δ(f−kf0) (D.5)
134 APPENDIX D. PERIODIC SIGNAL MODELING Then the spectral envelope of a such periodic signal can be written as:
S(f) =r0+ 2
We can define the even and odd parts of the cyclic correlation:
rkP = rkP,e +rkP,o, (D.9)
The influence on the spectrum is expressed as follow:
S(f) = Se(f) +So(f), (D.13) Figure D.4 shows the frequency selection of the even and odd parts. As the Fourier Transform is done with P points, with P the period of the signal, each point of the spectrum is a peak of the periodic signal and the Spectrum represents the spectral envelope if the signal is harmonic. If we define the fundamental frequency as the first harmonic, the even part cancels the odd harmonics and leaves the even harmonics unchanged and vice-versa for the odd part.
D.2. PERIODIC SIGNAL MODELING 135
0 1/P 2/P 3/P 4/P
0 0.2 0.4 0.6 0.8 1 1.2
Frequency
Spectrum
Even and Odd part Spectrum
Even part Odd part
Figure D.4: Even and odd parts of the spectrum.
D.2.1 Definition of the periodic signature
The signal is first resampled to a power of two in order to avoid problems when the even and odd part are computed and for having an integer period. Then the signal is cut into frames of lengthP, the periodic signature is expressed by itsgeneralized ACF :
RP =IDF T(|DF T(XP)|p) (D.18) where RP and XP are two matrices for which each column represents a period of the signal and its cyclic representation respectively:
XP = [x1 . . . xm] (D.19)
xm = [s(1+(m−1)P) . . . s(mP)]T (D.20)
whereT denote the transpose operator,m is the number of period in the analysed signal and x is a signal vector containingP samples.
As the harmonics of an audio signal are time varying and non perfectly harmonic, we need to have a robust estimate of the periodic signal. This signature is estimated as the principal vector of the eigenvalue decomposition of RP. We define u, the periodic signature, as the first column of U = SV D(RP).
Then the odd and even parts of the signature are computed as:
uP,ek = 1
2 (uPk +uPk +P 2
), (D.21)
uP,ok = 1
2 (uPk −uPk+P 2
), (D.22)
136 APPENDIX D. PERIODIC SIGNAL MODELING D.2.2 As a pitch detector
For estimating the pitch of the signal we reduce the set of fundamental frequencies to the first twelve frequencies of the first octave from a midi correspondance. For all of this sets we perform the algorithm described before and choose as candidate the one which maximize an energy criterium. Since the periodic signature is normalized in energy, we will work with its even part. The even part also represents the octave of the pitch, so we change the set of candidates to the previous octave. Working with the lower octave candidates didn’t reduces the set of octaves to the first one. When a candidate is chosen, we compute the energy of its Even T o Odd P arts Ratio (EOR); if it’s more than a threshold, then we decide that its true octave is the next one and we continue on the next octave by keeping, as periodic signature, the even part.
Since the energy of the periodic signature is normalised to one, the energy of the Even and Odd Part are bounded to 0.5, the chosen threshold is compared to the Even to Odd Parts Ratio and set to 10.
D.2.3 Simulation for the pitch estimation
For this simulation we have generated light inharmonic signals, in fact all the parameters are randomly generated. The Inharmonicity coefficient is set toβ = 10−5, so the frequen-cies follows as a rule fn = n f0p
1 + β n2. The amplitudes and phases are uniformly distributed in [0;1] and [0;2π] respectively. The amplitudes are also decreasing with the index and the sum of the amplitudes is normalized to 1. We have chosen the tessitura of the guitar for our analyse such that the set of midi code is [40;88].
40 45 50 55 60 65 70 75 80 85
Figure D.5: Pitch detection and Octave Selection for a synthetic signal.
Figure D.5 shows the result of the analysis: as expected, the notes are correctly inter-preted on the octave zero, and their true octaves are correctly found. The second possible
D.2. PERIODIC SIGNAL MODELING 137 candidate is also shown for each notes. As we can see for the first and a half octave, it has a semitone difference, while for the next octave it’s a perfect fourth difference (5 semitones upper).
D.2.3.1 Diagram for the pitch determination
The following diagram (Figure D.6) summarizes the algorithm for the pitch detection algorithm.
Resample to a power of 2 for the research of the octave
Figure D.6: Diagram of the pitch estimation algorithm
D.2.3.2 Illustration for the octave determination
In order to illustrate the good octave selection, we give in Figure D.7, D.8 and D.9 some details. The true octave is the 3rd one (from a midi correspondance) and the song is a guitar note, the lower note of a guitar is a E2 (82.4 Hz) and it begins at the second octave.
As mentioned before we begin at the octave−1, and, for the guitar, the octave−1 is the octave 1 of the midi reference. We apply the algorithm described before for finding the pitch. The first plot shows the signal (resampled), the second and third ones are the signal cut into small frames and its cyclic correlation respectively. Then the signature and the decomposition into its odd and even parts. The bottom part presents when we are at a lower octave than the good one, we have, in the periodic signature, more than one period and this gives more energy to the even part. This method is philosophically equivalent to the subharmonic-to-harmonic ratio one [162].
138 APPENDIX D. PERIODIC SIGNAL MODELING D.2.4 Application to a true signal
For this analysis we have recorded all the first 37 notes of the guitar (midicode 40 to 76) on an acoustic guitar. The notes are played with a guitar pick and the guitar was plugged and linked to an external soundcard. The analysis is made on the first 250msof the signal (including the attack). Note that the guitar was not perfectly tuned (impossible) and the used candidate are determined again by the midi reference frequency.
Figure D.5 shows the result of the analysis for the guitar, the result is not perfect, but we can see that if a note is not well detected its octave is false and the note found is the perfect fourth of the played note, the second candidat of the previous analysis. In this case the true note becomes the second choice. Note that the perfect fourth shares some harmonics in the even part but doesn’t share its fundamental frequency.
Figure D.7: Octave −2
Figure D.8: Octave −1
D.2. PERIODIC SIGNAL MODELING 139
Figure D.9: Good Octave
40 45 50 55 60 65 70 75
30 40 50 60 70
Note Detection and Octave Correction
Midi Code
Midi Code
Detected Notes Octave Correction 2nd candidat Solution
40 45 50 55 60 65 70 75 80
−1 0 1 2 3
Octave determination
Midi Code
Octave Number
Detected Octave Solution
Figure D.10: Pitch detection and Octave Selection for guitar.
140 APPENDIX D. PERIODIC SIGNAL MODELING D.2.5 Application to the Octave Problem
In this section we analyse the octave problem. The octave problem rises when a note and its octave are played together. They share the same periodicity and the even harmonics of the played notes are amplified by the harmonics of the Octave. For the analysis we assume that the fundamental frequencies are known. In spectral analysis there are, at least, two ways to estimate the even and odd frequencies. The first one consiste in finding all the peaks in the spectrum, by peak picking, and by paying attention not to miss some of them otherwise an odd harmonic can become an even harmonic and vice-versa, another point is the inharmonicity of the signal. To find the peaks we have to adjust, from one peak to the next one, the distance and searching a local maximum around it. The second method is equivalent to the proposed method; it consists in computing the spectra of the matrix XP, define before, and taking the average trough the time dimension. this is Welch’s periodogram, then the even harmonics are the even samples of the spectrum.
D.2.6 Note plus its Octave
In this case, a note is played with and without its octave, recorded in the same condition as before with an acoustic guitar. We compare the results of the proposed method with the first spectral techniques (with peak picking). The second spectral method, explain before gives very similar results to the proposed one (temporal), so we just show our proposed method. The results (Figure D.11) are poor for the two methods due to the coloration of the spectra.
40 45 50 55 60 65
−20
−15
−10
−5 0 5 10 15
Midi Code
dB
Even Part To Odd Part Ratio
Note
Note Plus Octave
Figure D.11: Octave problem, a note with its octave.
D.2. PERIODIC SIGNAL MODELING 141 We have decided to add in our framework another one preprocessing: for the rest of the simulation we worked in the prediction error of the signal. The signal is modeled as an autoregressive model of order ten. Morever, we defined that a note can’t be interpreted as its octave, but a note with its octave can be interpreted as the note by itself.
40 45 50 55 60 65
−4
−2 0 2 4 6 8 10 12
Midi Code
dB
Even Part To Odd Part Ratio
Note
Note Plus Octave
40 45 50 55 60 65
−10
−5 0 5 10 15
Midi Code
dB
Even To Odd Harmonic Ratio
Note
Note Plus Octave
Figure D.12: Octave problem in the prediction error, a note with its octave with the temporal method (top) and the spectral method (bottom).
The results (Figure D.12) are better for the two methods. The dashed line is the upper value of the notes alone, in the two cases we make one error.