HAL Id: hal-02158742
https://hal.archives-ouvertes.fr/hal-02158742
Submitted on 18 Jun 2019
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de
First Steps Towards an Organology Of Virtual Instruments In Computer Music
Jérôme Barthélemy, Alain Bonardi, Yann Orlarey, Serge Lemouton, Raffaele Ciavarella, Karim Barkati
To cite this version:
Jérôme Barthélemy, Alain Bonardi, Yann Orlarey, Serge Lemouton, Raffaele Ciavarella, et al.. First
Steps Towards an Organology Of Virtual Instruments In Computer Music. International Computer
Music Conference, 2010, New York, United States. pp.369-372. �hal-02158742�
FIRST STEPS TOWARD AN ORGANOLOGY OF VIRTUAL INSTRUMENTS IN COMPUTER MUSIC
Jérôme Barthélemy IRCAM
jerome.barthelemy@ircam.fr
Yann Orlarey Grame orlarey@grame.fr Alain Bonardi
IRCAM alain.bonardi@ircam.fr
Serge Lemouton IRCAM
serge.lemouton@ircam.fr Raffaele Ciavarella
IRCAM
raffaele.ciavarella@ircam.fr
Karim Barkati Grame karim.barkati@free.fr ABSTRACT
In this paper, we will first take assess of 25 years of interactive real-time music, and introduce the problem of preservation of this music for the future generations, that is to say its ability to be re-performed, and not only to preserve the recordings. We present the state of the art in the field of active preservation of real-time works. We then give an overview of the solutions developed by IRCAM and its partners Grame, Armines ParisTech and CIEREC, in the framework of the ASTREE project, and explain the possibilities envisioned in a case study that is En Echo by Philippe Manoury.
1. INTRODUCTION
Thanks to electronic and computerized systems, contemporary composers have been able for half a century to overcome their usual practice by composing sounds and not only notes. This has led to a new kind of instrument- making, and raised a new issue: music instruments in the past would make up a stable cultural substrate, enabling to build a tradition; on the contrary, recent mixed works combining acoustic instruments and sound transformation processes have to put up with the accelerated obsolescence of digital technologies.
Facing instability and technological dependency, first approaches were based on the preservation of all devices and all necessary knowledge for further performances.
They are implicitly based on the risky possibility of return to the composer or any person implied in the creation process. Current efforts deal with structured documentation according to such international standards as OAIS [ISO standard 14721:2003]. The ideal target would be to capture the intentions of a composer in a work, independently from the technology used.
In this paper, we will first take assess of 25 years of interactive real-time music. We present the state of the art in the field of active preservation of real-time works. We then give an overview of the solutions developed by IRCAM and its partners in the framework of the ASTREE project and detail our case study based on En Echo by Manoury.
2. AFTER 25 YEARS OF INTERACTIVE REAL- TIME WORKS
First interactive works associating instrumentalists and electronic real-time modulation of their parts go back to the mid 1980s. Several musical configurations have been explored at IRCAM: from the voice-machine duet with for instance En Echo by Philippe Manoury (1993-1994) up to works for soloists, ensemble and electronics as Répons by Pierre Boulez (1981-1988) After 25 years of creation, a repertoire is emerging, requiring preservation and durability.
2.1. A new dependency to material conditions of performances
Interactive real-time works bring the dependency to material conditions to an unknown degree. For instance, the re-performance of Diadèmes by Marc-André Dalbavie in 2008, 23 years after its creation, raised a major issue with the obsolescence of the TX816 FM synthesizers initially used. TX816 synthesizers are no longer manufactured, the ones IRCAM possesses are nearly out of order; the composer has tried several software emulators and none of them gave a satisfying result.
It is a new situation: these works can no longer lean on stable cultural paradigms from what we call “occidental classical music” in a broad meaning, such as notation and organization in instrumental families. Each work becomes a system. These works constitute a repertoire, a part of
IRCAM patrimony, and a part of electro-acoustic music history. IRCAM has made a commitment to composers to ensure the preservation of the commissioned works. More generally, these musical pieces are cultural objects belonging to the tradition of occidental serious music. The topic of preserving these works, regardless of their intrinsic qualities is crucial because it is a whole chapter of occidental music recent history that could disappear as a consequence of the bad transmissibility of digital media.
2.2. A new instrument-making without organology In terms of musical functions, computerized modules inside interactive real-time works take a stand between the score paradigm and the intrument one. The emerging NIME (New Interfaces for Musical Expression) community has shown a large number of inventions and a new instrument-making. But it is not yet possible to build organological classifications of these systems. The approaches in classical organology intersecting families of timbres and families of “voices” are no longer relevant in interactive music.
3. ACTIVE PRESERVATION OF REAL-TIME WORKS
3.1. State of the art in the field of active preservation of real-time works: emulation and migration
Preservation of performance arts including electronics has only been studied for a few years. The Mustica Research Initiative, an international project coordinated by the University of Technology of Compiègne, has built on a research collaboration on the topic of contemporary music preservation between two french contemporary music institutions (IRCAM & INA) between 2003 and 2004 [1].
Emulation is certainly one of the most difficult approaches of re-performance. Bullock and Coccioli [3]
provide an interesting example by composer Jonathan Harvey: his piece Madonna of winter and spring, for large orchestra and three synthetisers, requires a Yamaha TX816 for sound synthesis. The authors are planning to emulate the synthetiser with PureData patches.
Migration is the most widespread activity to achieve re- performance. All institutions in the field of electronic arts face migration issues. At IRCAM, important pieces using Next computers were moved to Macintosh machines at the end of the 1990s. Miller Puckette [6] has put the emphasis on the necessity of standard and open formats to store all files used in the process (patches in ASCII text format, documentation in plain text, audio contents as basic WAV files).
3.2. Virtualization: the concept
In terms of software preservation, virtualization is the expression of digital objects in a form independant from
hardware or software dependencies. This goal, easy to express, if obviously very difficult to achieve, since the ability to understand the expressions of the digital objects depends on the knowledge of the reader. In the ASTREE project, we plan different levels of virtualization for real- time processes, that are intended to different user communities.
3.3. Virtualization of real-time processes: three different possible levels
3.3.1. First level using well defined syntax and semantics A first level of virtualization for digital objects can be achieved by generating an expression of the original object in a form where syntax and semantics are well defined.
In the previous CASPAR project [4], we defined such an expression by using the well defined XML syntax, and defining the adequate semantics, by extracting it from current existing documentation. In the ASTREE project, we keep this level of virtualization, but we move a step forward by expressing it using the FAUST language [5].
This language has some very interesting characteristics, compared to the XML expression:
- it is concise, in such a manner that the program can be expressed in a very simple ASCII form, not verbose (as XML), and understandable by a reader accustomed to FAUST syntax and semantics;
- it is sufficiently expressive to encode the synchronous part of the original objects, being a language devoted to DSP (Digital Signal Processing).
It is to be noticed that, regarding the asynchronous part of the original objects, ASTREE made the choice not to handle it, the reason of this being that the mathematical formalism envisioned to document the process is not at all efficient for expressing concepts related to time, in the musical meaning of the term.
3.3.2. Second level: using a preservable formalism The second level of virtualization of real-time processes we planned is their description using a preservable formalism. The mathematical formalism has made the proof of its sustainability, and thus, the idea is to preserve the mathematical expression of the original objects, printed on paper, or in the best sustainable form of digital document available today, like PDF-A (ISO standard 19005-1).
3.3.3. Third level: toward an organology.
The third level of virtualization considered is the description of real-time processes using a commonly accepted terminology, that should be developed by first organizing a repository for the objects we intend to manage, and second building knowledge about these
objects (an « ontology » of the domain). We should use several methods in the knowledge engineering field: data mining, collaborative methods, etc. Moreover, and even if probably the documentation using a mathematical formalism is the better form of long term preservation for the objects, for middle-term or short-term preservation, some more pragmatic methods can be envisioned. At the first sight, some objects we manage are well-known, and there is no need to re-implement these objects, as equivalent objects have a lot of chances to exist in the future.
This level of virtualization is aimed to musicians, and users of these objects, that have no knowledge of FAUST syntax or semantics, no knowledge about mathematics, but have knowledge of music and its terminology.
4. CURRENT STATE OF VIRTUALIZATION IN THE ASTREE PROJECT
At this stage in the ASTREE project, and after a year of work, we can now demonstrate at least partially the approach explained above. We start from an object – here below a very simple example of additive synthesis – implemented in the Max/MSP software. We then translate it into a FAUST expression, and generate a self-sufficient documentation. This is possible due to two different software prototypes: a translator first, and a documentation compiler.
4.1. Virtualization: the original object
Figure 1 is a screenshot of the original Max/MSP patch,
“addsynth”:
Figure 1. An original Max/MSP object.
4.2. Virtualization: the FAUST expression
Here is the corresponding expression of the same object, in FAUST:
// Process process =
(((osc(330),osc(330.2):+),(osc(440),osc(440.33):+):+),(osc(587),osc(587.25):+):+);
Figure 2. the corresponding FAUST expression
One should notice some important facts:
- This expression is compilable, that is to say that a C program can be compiled from it, and compiled in an executable form (this C program can be considered an other form of “virtualization”).
- In order to read and understand this expression, one should know the syntax and the semantics of FAUST language, but one does not depend on any software or hardware, that expression being expressed in simple ASCII text.
4.3. Virtualization: the documentation
Figure 3 shows an extract of the automatically generated documentation, precisely the mathematical expression of the Max/MSP object.
Figure 3. An extract of the documentation.
In the current documentation, there are also notes about these, remaining mathematical formulas that are not shown here, a block-diagram expression (not shown), and original FAUST code.
This documentation is stored in PDF-A format, that is the most advanced standard for long-term preservation, and provides today the better guaranties of durability.
5. A CASE STUDY: EN ECHO, BY PHILIPPE MANOURY
To show the relevance of these approaches, we have to apply them to musical works that are interesting both on an artistic point of view and on a validation one. We have selected En Echo for solo voice and live electronics (1993- 1994) by Manoury as a first test work mainly because Manoury organizes his real-time processes as an orchestra, making easy to identify both interesting parts of the patch and precise moments when they are used. Applying our static analysis tool [2], we have highlightened synchronous modules that could be reimplemented using ASTREE methodology. We give up asynchronous
handling of the piece using qlists, and we focus more specifically on:
! The process of the voice; E_Pitch (pitch, sibilant and silence recognition) and E_lpc (formant detection) modules seem interesting, but they require externals: fiddle~ by Miller Puckette, zerocross~ in the MSP Jimmies library, and formant~ object by Serge Lemouton. Though the source code can be obtained, their FAUST description may be not that easy.
! Sound generation; we are particularly interested in three modules: E_Pafs (Phase Aligned Formants), E_noise (noise generation) and E_harms (harmonizer). They are lightly dependent from the ISPW library, but that seems easy to overcome.
6. CONCLUSION AND FUTURE WORK Being today at nearly the half of the ASTREE project, we can demonstrate that we can express the original Max/MSP objects into several FAUST expressions, each of these being a different level of virtualization of the same object:
an expression using the FAUST syntax and semantics, a documentation using mathematical formalism, and also an expression in the form of a C++ program (or even of an executable...).
6.1. Validation
In the ASTREE project, the demonstration of the adequacy of the mathematical documentation to the preservation of real-time process has to be shown. To this end, we intend to ask persons from different communities to re-implement the original process, using their own preferred environment (even if it is Max/MSP!). This way, we could have different re-implementations of the same object: one using the original platform, and several using different platforms (such as Matlab, CSound...). The reimplementation should be validated by comparing the outputs of the different objects, applied to the same input, and using criteria established on one hand on audio engineering criteria, and on the other hand on extraction of low-level descriptors.
We will also demonstrate the adequacy of the approach by applying it on En Echo by Philippe Manoury, re- implement some modules, and propose to the composer himself to validate the new implementation.
6.2. Towards an organology
At this stage of the project, we are in the situation of being able to construct the repository we need. We have at our disposal a lot of original Max/MSP objects, and we are almost ready to apply our translation to them in order to obtain FAUST expressions, and the corresponding documentation.
The amount of work towards an organology should not be underestimated. Not only the adequate descriptors have to be extracted, not only the adequate data mining methods have to be applied, but also a proactive methodology is to be defined: the “organology” we expect can not simply emerge from data. We will use an empirical methodology:
adequate theories have to be defined, and validated (or rejected) on the basis of the analysis of the repository. It is not the purpose of this article to expose these theories.
This is certainly a long way, and we will probably not achieve it completely in the ASTREE project, but merely will make some steps forward in that direction.
7. ACKNOWLEDGEMENTS
This work was possible partially due to a grant provided by French National Research Agency, under the number 08- CORD-003-01.
This work was possible due to the collaboration of partners of the ASTREE project: IRCAM, Grame, Armines ParisTech, and CIEREC.
8. REFERENCES
[1] Bachimont, B., Blanchette, J.-F., Gerszo, A., Swetland, A., Lescurieux, O., Morizet-Mahoudeaux, P., Donin, N., Teasley, J., “Preserving Interactive Digital Music: A Report on the MUSTICA Research Initiative”, in Proceedings of the Third International Conference on WEB Delivering of Music (WEB’03), Leeds, England, 2003.
[2] Bonardi, A., Barthélemy, J., Ciavarella, R., Boutard, G., “Will Software Modules for Performing Arts be Sustainable? ”, in Proceedings of the 14th IEEE Mediterranean Electrotechnical Conference (MELECON'08), Ajaccio, France, 2008.
[3] Bullock, J., Coccioli, L., “Modernising Live Electronics Technology in the Works of Jonathan Harvey”, in Proceedings of the 2005 International Computer Music Conference, Barcelona, Spain, 2005.
[4] Kia, N., Tran, V., Bee, O., Bonardi, A., Barthélemy, J., Giaretta, D., 2007, “Digital Preservation of Interactive Multimedia Performances”, in Proceedings of the 2007 International Computer Music Conference, Copenhagen, 2007.
[5] Orlarey, Y., Fober, D., Letz, S., “An Algebra for Block Diagram Languages”, in Proceedings of the 2002 International Computer Music Conference, Gothenburg, 2002.
[6] Puckette, M., “New Public-Domain Realizations of Standard Pieces for Instruments and Live Electronics”, in Proceedings of the 2004 International Computer Music Conference, Miami, 2004.
