Instrument Recognition
4.5 Future Directions
The problem of instrument recognition is still open. Even the identification of instruments in isolated tones is far from being a mature technology. Hence, the research is still in a prospective phase, which means that there are many directions to be taken and many solutions to be tested.
The task of feature extraction has been intensely studied for a long time due to the importance it has in characterizing audio signals. Because of that, there is not much room for improvement. However, especially in the case of polyphonic signals, it may still be possible to create new features capable of extracting some kind of information that no other feature can capture.
The classifiers used by most algorithms are also well established, making significant advances even more difficult than in the case of the feature extrac-tion. Moreover, many studies that compared a number of classifiers under the same conditions reveal that the classifiers actually have similar performances, which indicates that this may not be among the most important factors that influence the accuracy of the methods.
A better candidate for improvement may be the preprocessing of the signal.
This stage of the algorithm aims to modify the signal in such a way it becomes more prone to the subsequent processing. Since musical signals are not well behaved, in the sense that they do not follow clear instrument-related rules, a novel preprocessing stage will have to incorporate some mechanism to make the signals more predictable and, more importantly, to make the characteris-tics of each instrument stand out. Clearly this is not an easy task, but given the ability that human listeners have to perceive even the slightest particular characteristics of a given instrument, such an objective is not infeasible.
It is important to notice that the feature/classifier combination is not the only possible way to recognize instruments. An interesting approach that has been already explored in some studies is the template matching. The main goal of this kind of strategy is to find, for each possible instrument, one or more representations (templates) that are consistently valid despite all the variability between instrument samples. Those templates have also to deal properly with the entire frequency range of each instrument. If the templates are really representative, they will match well with any representation ex-tracted from the signal to be classified. Again, this is not an easy task, but studies performed so far [40, 84] indicate that this option has good potential.
In the specific case of polyphonic signals, the great difficulty lies in the cross-interference caused by simultaneous instruments. In this case, there are two main possible directions for future research. The first one consists in breaking the polyphonic problem into a number of monophonic ones. The problem with this option is that it depends on advances in sound source separation, which is a very difficult problem by itself, especially if there are more instruments than channels. To make things harder, to be useful in this context the source separation needs to be close to perfect, because while the instrument recognition is difficult with its temporal and spectral contents intact, it is nearly impossible if the contents are too distorted. The second option is to explore the temporal and spectral disjointness among instruments that usually occur in any signal. The idea here is identify the regions in time and/or frequency where a given signal is isolated, and then use only those clean parts to perform the identification. In the time domain, this implies in finding isolated notes, and in the frequency domain, the objective is identify partials that do not collide with any other one, and filter the signal to eliminate the remaining mixed partials. Some studies have already been carried out [4, 49], with promising results.
As can be seen, there are many possible options to be investigated and developed. Thare are many aspects of the problem that are still not well known, including the maximum information that can be extracted from a musical signal and what is the best way to explore such an information. Future studies will have the responsibility to bring those important questions closer to an answer.
Bibliography
[1] University of Iowa musical instrument samples database.
[2] G. Agostini, M. Longari, and E. Pollastri. Musical instrument timbres classification with spectral features. EURASIP Journal Applied Signal Processing, 2003:5–14, 2003.
[3] J.G.A. Barbedo, A. Lopes, and P.J. Wolfe. Empirical methods to de-termine the number of sources in single-channel musical signals. IEEE Transactions on Audio, Speech and Language Processing, 17:1435–1444, 2009.
[4] J.G.A. Barbedo and G. Tzanetakis. Musical instrument classification using individual partials.IEEE Transactions on Audio, Speech and Lan-guage Processing, 19:111–122, accepted for publication.
[5] J.P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M.B.
Sandler. Adaptive Control Processes: Guided Tour. Princeton University Press, New Jersey, 1961.
[6] J.P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M.B.
Sandler. A tutorial on onset detection in music signals. IEEE Transac-tions Speech and Audio Processing, 13:1035–1047, 2005.
[7] E. Benetos, M. Kotti, and C. Kotropoulos. Musical instrument classi-fication using non-negative matrix factorization algorithms and subset feature selection. InProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 221–224, 2006.
[8] J.C. Brown. Computer identification of musical instruments using pattern recognition with cepstral coefficients as features.Journal of the Acoustical Society of America, 105:1933–1941, 1999.
[9] J.C. Brown, O. Houix, and S. McAdams. Feature dependence in the automatic identification of musical woodwind instruments. Journal of the Acoustical Society of America, 109:1064–1072, 2001.
[10] J.J. Burred, A. Robel, and T. Sikora. Dynamic spectral envelope model-ing for timbre analysis of musical instrument sounds.IEEE Transactions on Audio, Speech and Language Processing, 18:663–674, 2010.
[11] N. Chetry and M. Sandler. Linear predictive models for musical instru-ment identification. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 173–176, 2009.
[12] G. Costantini, A. Rizzi, and D. Casali. Recognition of musical instru-ments by generalized min-max classifiers. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, pages 555–564, 2003.
[13] J.D. Deng, C. Simmermacher, and S. Cranefield. A study on feature analysis for musical instrument classification.IEEE Transaction Systems Man Cybernetics—Part B: Cybernetics, 38:429–438, 2008.
[14] J.S. Downie. The music information retrieval evaluation exchange (2005–
2007): A window into music information retrieval research. Acoustical Science and Technology, 29:247–255, 2008.
[15] J. Eggink and G.J. Brown. A missing feature approach to instrument identification in polyphonic music. In Proceedings of the IEEE Inter-national Conference on Acoustics, Speech and Signal Processing, pages 553–556, 2003.
[16] J. Eggink and G.J. Brown. Instrument recognition in accompanied sonatas and concertos. InProceedings of the IEEE International Confer-ence on Acoustics, Speech and Signal Processing, pages 217–220, 2004.
[17] J. Eggink and A. Klapuri. Musical instrument recognition using cepstral coefficients and temporal features. In Proceedings of the IEEE Inter-national Conference on Acoustics, Speech and Signal Processing, pages 753–756, 2000.
[18] C. Elliott. Attacks and releases as factors in instrument identification.
Journal Research in Music Education, 23:35–40, 1975.
[19] A. Eronen. Automatic musical instrument recognition. Master’s thesis, Tampere University of Technology, Finland, 2001.
[20] A. Eronen. Comparison of features for music instrument recognition. In Proceedings of the IEEE Workshop on Application Signal Processing to Audio and Acoustics, pages 19–22, 2001.
[21] A. Eronen. Musical instrument recognition using ICA-based transform of features and discriminatively trained HMMs. In Proceedings of the International Symposium on Signal Processing Application, 2003.
[22] S. Essid, G. Richard, and B. David. Efficient musical instrument recog-nition on solo performance music using basic features. InProceedings of the AES International Conference, 2004.
[23] S. Essid, G. Richard, and B. David. Musical instrument recognition based on class pair-wise feature selection. InProceedings of the International Conference on Music Information Ret., 2004.
[24] S. Essid, G. Richard, and B. David. Hierarchical classification of musical instruments on solo recordings. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 817–820, 2006.
[25] S. Essid, G. Richard, and B. David. Instrument recognition in polyphonic music based on automatic taxonomies. IEEE Transactions on Audio, Speech and Language Processing, 14:68–80, 2006.
[26] S. Essid, G. Richard, and B. David. Musical instrument recognition by pair-wise classification strategies. IEEE Transactions on Audio, Speech and Language Processing, 14:1401–1412, 2006.
[27] D. Fragoulis, C. Papaodysseus, M. Exarhos, G. Roussopoulos, T. Panagopoulos, and D. Kamarotos. Automated classification of pi-anoguitar notes. IEEE Transactions on Audio, Speech and Language Processing, 14:1040–1050, 2006.
[28] M. Goto. Development of the RWC music database. In Proceedings of the International Congress on Acoustics, pages 553–556, 2004.
[29] M. Goto, H. Hashigushi, T. Nishimura, and R. Oka. RWC music database:
Popular, classical, and jazz music databases. In Proceedings of the In-ternational Conference on Music Information Retrieval, pages 287–288, 2002.
[30] S. Gunasekaran and K. Revathy. Fractal dimension analysis of audio sig-nals for Indian musical instrument recognition. InProceedings of the In-ternational Conference on Audio, Language and Image Processing, pages 257–261, 2008.
[31] I. Guyon and A. Elisseeff. An introduction to variable and feature selec-tion. Journal Machine Learning Research, 3:1157–1182, 2003.
[32] P. Herrera-Boyer, A. Klapuri, and M. Davy. Automatic classification of pitched musical instrument sounds. InSignal Processing Methods for Music Transcription, pages 166–200. Springer, New York, 2006.
[33] P. Herrera-Boyer, G. Peeters, and S. Dubnov. Automatic classification of musical instrument sounds. Journal of New Music Research, 32:3–21, 2003.
[34] M. Ihara, S.-I. Maeda, and S. Ishii. Instrument identification in mono-phonic music using spectral information. In Proceedings of the IEEE International Symposium on Signal Processing and Information Technol-ogy, pages 595–599, 2007.
[35] P. Jincahitra. Polyphonic instrument identification using independent subspace analysis. InProceedings of the IEEE International Conference on Multimedia and Expo, pages 1211–1214, 2004.
[36] C. Joder, S. Essid, and G. Richard. Temporal integration for audio clas-sification with application to musical instrument clasclas-sification. IEEE Transactions on Audio, Speech and Language Processing, 17:174–186, 2009.
[37] I. Kaminskyj and A. Materka. Automatic source identification of mono-phonic musical instrument sounds. InProceedings of the IEEE Interna-tional Conference on Neural Networks, pages 189–194, 1995.
[38] I. Kaminskyj and P. Voumard. Enhanced automatic source identifica-tion of monophonic musical instrument sounds. In Proceedings of the Australian, New Zealand Conference on Intelligent Information Systems, pages 189–194, 1995.
[39] I. Kaminskyj and T. Czaszejko. Automatic recognition of isolated mono-phonic musical instrument sounds usingk-NNC. Journal Intelligent In-formation Systems, 24:199–221, 2005.
[40] K. Kashino and H. Murase. A sound source identification system for en-semble music based on template adaptation and music stream extraction.
Speech Communication, 27:337–349, 1999.
[41] R.A. Kendall. The role of acoustic signal partitions in listener catego-rization of musical phrases. Music Perception, 4:185–214, 1986.
[42] T. Kitahara. Computational musical instrument recognition and its ap-plication to content-based music information retrieval. PhD thesis, Kyoto University, Japan, 2007.
[43] T. Kitahara, M. Goto, K. Komatani, T. Ogata, and H.G. Okuno. Instru-ment identification in polyphonic music: Feature weighting with mixed sounds, pitch-dependent timbre modeling, and use of musical context. In Proceedings of the International Conference on Music Information Re-trieval, pages 558–563, 2005.
[44] T. Kitahara, M. Goto, K. Komatani, T. Ogata, and H.G. Okuno. In-strogram: A new musical instrument recognition technique without using onset detection nor F0 estimation. In Proceedings of the IEEE Inter-national Conference on Acoustics, Speech and Signal Processing, pages 229–232, 2006.
[45] T. Kitahara, M. Goto, K. Komatani, T. Ogata, and H.G. Okuno. Instru-ment identification in polyphonic music: Feature weighting to minimize influence of sound overlaps.EURASIP Journal Applied Signal Processing, 2007.
[46] T. Kitahara, M. Goto, and H.G. Okuno. Category-level identification of non-registered musical instrument sounds. InProceedings of the IEEE In-ternational Conference on Acoustics, Speech and Signal Processing, pages 253–256, 2004.
[47] T. Kitahara, M. Goto, and H.G. Okuno. Pitch-dependent identification of musical instrument sounds. Applied Intelligence, 23:267–275, 2005.
[48] A. Klapuri. Sound onset detection by applying psychoacoustic knowledge.
InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 3089–3092, 1999.
[49] B. Kostek. Musical instrument classification and duet analysis employing music information retrieval techniques.Proceedings of the IEEE, 92:712–
729, 2004.
[50] B. Kostek. Intelligent musical instrument sound classification. In Perception-Based Data Processing in Acoustics, pages 39–186. Springer-Verlag, New York, 2005.
[51] B. Kostek and A. Czyzewski. Representing musical instrument sounds for their automatic classification. Journal Audio Eng. Society, 49:768–785, 2001.
[52] O. Kramer and T. Hein. Stochastic feature selection in support vector machine based instrument recognition. InKI 2009: Advances in Artificial Intelligences, number 5803 in Lecture Notes in Computer Science, pages 727–734. Springer-Verlag, New York, 2009.
[53] A.G. Krishna and T. V. Sreenivas. Music instrument recognition: From isolated notes to solo phrases. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 265–268, 2004.
[54] E. Kubera and Z.W. Ras. Identification of musical instruments by fea-tures describing sound changes in time. InAdvances in Intelligent Infor-mation Systems, pages 357–366. Springer-Verlag, New York, 2010.
[55] P.S. Lampropoulou, A.S. Lampropoulos, and G.A. Tsihrintzis. Musical instrument category discrimination using wavelet-based source separa-tion. In New Directions in Intelligent Interactive Multimedia, volume 142, pages 127–136. Springer-Verlag, New York, 2008.
[56] D. Little and B. Pardo. Learning musical instruments from mixtures of audio with weak labels. In Proceedings of the International Conference on Music Information Retrieval, pages 127–132, 2003.
[57] A. Livshin and X. Rodet. The importance of cross database evaluation in sound classification. InProceedings of the International Conference on Music Information Retrieval, 2003.
[58] A. Livshin and X. Rodet. Instrument recognition beyond separate notes—
Indexing continues recordings. InProceedings of the International Com-puting Music Conference, 2004.
[59] A. Livshin and X. Rodet. Purging musical instrument sample databases using automatic musical instrument recognition methods. IEEE Trans-actions on Audio, Speech and Language Processing, 17:1046–1051, 2009.
[60] R. Loughran, J. Walker, M. ONeill, and M. OFarrell. Musical instru-ment identification using principal component analysis and multi-layered perceptrons. In Proceedings of the International Conference on Audio, Language and Image Processing, pages 643–648, 2008.
[61] K.D. Martin. Sound-source recognition: A theory and computational model. PhD thesis, Massachusetts Institute of Technology, 1999.
[62] K.D. Martin and Y.E. Kim. Musical instrument identification: A pattern-recognition approach. In Proceedings of the Meeting of the Acoustics Society America, 1998.
[63] L.G. Martins, J.J. Burred, G. Tzanetakis, and M. Lagrange. Polyphonic instrument recognition using spectral clustering. In Proceedings of the International Conference on Music Information Retrieval, 2007.
[64] G. Mazarakis, P. Tzevelekos, and G. Kouroupetroglou. Musical instru-ment recognition and classification using time encoded signal processing and fast artificial neural networks. InAdvances in Artificial Intelligence, number 3955 in Lecture Notes in Artificial Intelligence, pages 246–255.
Springer-Verlag, New York, 2006.
[65] S. McAdams, J. W. Beauchamp, and S. Meneguzzi. Discrimination of musical instrument sounds resynthesized with simplified spectrotemporal parameters. Journal Acoustics Society America, 105:882–897, 1999.
[66] J. McKay, M. Gainza, and D. Barry. Evaluating ground truth for ADRess as a preprocess for automatic musical instrument identification. InPres.
AES Convention, page 7816, 2009.
[67] A.B. Nielsen, S. Sigurdsson, L.K. Hansen, and J. Arenas-Garca. On the relevance of spectral features for instrument classification. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 485–488, 2007.
[68] F. Opolko and J. Wapnick. McGill-Queen’s University Master Samples.
McGill University, Press, Canada, 1987.
[69] T.H. Park. Towards automatic musical instrument timbre recognition.
PhD thesis, Princeton University, New Jersey, 2004.
[70] S.-C. Pei and N.-T. Hsu. Instrumentation analysis and identification of polyphonic music using beat-synchronous feature integration and fuzzy clustering. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 169–172, 2009.
[71] D. Piccoli, M. Abernethy, S. Rai, and S. Khan. Applications of soft computing for musical instrument classification. In AI 2003: Advances in Artificial Intelligence, 16th Austrian Conference on Artificial Intelli-gence, number 2903 in Lecture Notes in Computer Science, pages 878–
889. Springer-Verlag, New York, 2003.
[72] C. Pruysers, J. Schnapp, and I. Kaminskyj. Wavelet analysis in musi-cal instrument sound classification. InProceedings of the International Symposium on Signal Processing Application, 1998.
[73] X. Rodet and F. Jaillet. Detection and modeling of fast attack transients.
Proceedings of the International Computing Music Conference, 2001.
[74] C. Simmermacher, D. Deng, and S. Cranefield. Feature analysis and classification of classical musical instruments: An empirical study. In Advances of Data Mining, volume 4065, pages 444–458. Springer-Verlag, New York, 2006.
[75] P. Somerville and A.L. Uitdenbogerd. Multitimbral musical instrument classification. In Proceedings of the International Symposium on Com-puting Science Application, pages 269–274, 2008.
[76] A. Srinivasan, D. Sullivan, and I. Fujinaga. Recognition of isolated instru-ment tones by conservatory students. InProceedings of the International Conference on Music Perception and Cognition, pages 17–21, 2002.
[77] B. Tan and D. Sen. The use of the attack transient envelope in instrument recognition. InProceedings of the Australian International Conference on Speech and Science Technology, pages 489–493, 2006.
[78] A. Tindale, A. Kapur, and I. Fujinaga. Towards timbre recognition of percussive sounds. InProceedings of the International Computing Music Conference, pages 592–595, 2004.
[79] E. Vincent and X. Rodet. Instrument identification in solo and ensemble music using independent subspace analysis. In Proceedings of the In-ternational Conference on Music Information Retrieval, pages 576–581, 2004.
[80] E. M. von Hornbostel and C. Sachs. Systematik der musikinstrumente.
Zeitschrift, 46:553–590, 1914.
[81] A.A. Wieczorkowska and E. Kubera. Identification of a dominating in-strument in polytimbral same-pitch mixes using SVM classifiers with non-linear kernel. Journal Intelligent Information System, published OnLine, 2009.
[82] A.A. Wieczorkowska and A. Kubik-Komar. Application of discrimi-nant analysis to distinction of musical instruments on the basis of se-lected sound parameters. In Man-Machine Interactions, number 59 in Advances in Intelligent and Soft Computing, pages 407–416. Springer-Verlag, Berlin, 2009.
[83] A.A. Wieczorkowska, J. Wroblewski, P. Synak, and D. Slezak. Appli-cation of temporal descriptors to musical instrument sound recognition.
Journal Intelligent Information System, 21:71–93, 2003.
[84] K. Yoshii, M. Goto, and H.G. Okuno. Drum-sound recognition for poly-phonic audio signals by adaptation and matching of spectrogram tem-plates with harmonic structure suppression. IEEE Transactions on Au-dio, Speech and Language Processing, 15:333–345, 2007.
[85] L. Yu and H. Liu. Efficient feature selection via analysis of relevance and redundancy. Journal Machine Learning Research, 5:1205–1224, 2004.