Thesis
Reference
Musical minds : experience induced changes in music processing revealed by electrical neuroimaging and behavioural approaches
JAMES, Clara
Abstract
Dans la musique tonale occidentale, les phrases musicales se terminent par une résultante spécifique, hautement attendue. Nous avons réalisé diverses études sur le traitement d'irrégularités musicales syntaxiques en fonction de l'expérience musicale. Les réponses comportementales ont démontré que des enfants tout-venants manifestent déjà dès 6 ans des connaissances implicites de la syntaxe musicale. Pourtant, l'identification de violations harmoniques subtiles dans un contexte musical complexe s'avère difficile pour des enfants fréquentant l'école primaire ainsi que pour des adultes sans formation musicale. Ces violations fines sont parfaitement identifiées par des musiciens professionnels et évoquent chez eux des réponses électroencéphalographiques sélectives rapides et intenses. Grâce à des analyses statistiques des sources présumées de ces réponses électroencéphalographiques, une dynamique spécifique de ces activations cérébrales adaptatives chez les musiciens experts a pu être mise en évidence dans des aires temporales médiales droites, insulaires, frontales et pariétales.
JAMES, Clara. Musical minds : experience induced changes in music processing revealed by electrical neuroimaging and behavioural approaches. Thèse de doctorat : Univ. Genève et Lausanne, 2008, no. Neur. 17
URN : urn:nbn:ch:unige-881
DOI : 10.13097/archive-ouverte/unige:88
Available at:
http://archive-ouverte.unige.ch/unige:88
DOCTORAT EN NEUROSCIENCES des Universités de Genève et de Lausanne
UNIVERSITE DE GENEVE, FPSE
Professeur Claude-Alain Hauert, directeur de thèse
MUSICAL MINDS:
EXPERIENCE INDUCED CHANGES IN MUSIC PROCESSING REVEALED BY ELECTRICAL NEUROIMAGING AND BEHAVIOURAL APPROACHES
THESE Présentée à la
FPSE
de l’Université de Genève pour obtenir le grade de Docteure en Neurosciences
par
Clara JAMES des
Pays-Bas Thèse N° 17
Genève Mars 2008
SCIENCES DE L'EDUCATION
James, C.E., Britz, J., Vuilleumier, P., Hauert, C.-A., Michel, C.M. Plasticity in right limbic structures mediates harmony incongruity processing in musical experts. Under review.
James, C.E., Michel, C.M., Britz, J., Vuilleumier, P., Hauert, C.-A. Rhythm evokes action: processing of metric anomalies in expressive music by experts and laymen revealed by electrical neuroimaging. Under review.
"Rhythm and harmony enter most powerfully into the innermost part of the soul and lay forcible hand upon him, bearing grace with them, so making graceful him who is
rightly trained"
Plato (428 BC - 347 BC) The Republic, Book III
In western tonal music, musical phrases end with an explicit consequent, which is highly expected. As such expectation is a consequence of musical background, processing of incongruities of musical grammar might be a function of musical experience. We conducted several studies on the processing of irregularities of musical syntax as a function of experience. Adults with different levels of musical expertise and primary school children of various ages appraised expressive musical stimuli that contained different grammatical incongruities at closure, interspersed with regular endings. Behavioural responses showed that, as a result of mere exposure, implicit learning of musical syntax manifests already in six year old children and progressively develops with age. However, judgement of refined harmonic syntactic violations in a complex polyphonic musical context did not develop with age in the primary school children interrogated here, and also untrained adults could barely detect them. In professional musicians, who identified these violations perfectly, they evoked selective rapid and strong electroencephalographic responses. If such a reaction is due to neuroplasticity as a consequence of training, it should be correlated with distinctive event-related potential (ERP) components deriving from differential activations in sensory, motor and/or cognitive function related brain areas at specific moments after the presentation of the incongruent closure. We could demonstrate striking differences in behavioural and ERP responses as a function of experience at all levels of analysis, and moreover, were able to reveal, by means of statistical analysis of putative sources of the ERPs, specific brain activations in response to the different grammatical anomalies in musical experts in right medial temporal, insular, frontal and parietal areas of the brain.
It appears that implicit learning has limits, and to make further progress in musical processing explicit and intensive training is required. This intensive and explicit training inevitably adapts the brain.
Dans la musique tonale occidentale, les phrases musicales se terminent par une résultante spécifique, hautement attendue. Une telle attente découlant de l'expérience musicale de l'individu, le traitement d’une incongruité musicale syntaxique devrait donc dépendre de l'expérience musicale. Nous avons réalisé diverses études sur le traitement des irrégularités de la syntaxe musicale en fonction de l'expérience musicale. Des adultes de niveaux d'expertise différents ainsi que des enfants d'âge variable fréquentant l'école primaire ont évalué des stimuli musicaux expressifs contenant ou non des incongruités grammaticales diverses à leurs terminaisons. Les réponses comportementales ont montré que des enfants tout-venants de 6 ans déjà manifestent des connaissances implicites de la syntaxe musicale, suite à une simple exposition, qui se développent ensuite avec l'âge. Pourtant, le jugement de violations harmoniques subtiles dans un contexte musical polyphonique complexe ne se développe pas avec l'âge dans le groupe d'enfants interrogés ici et des adultes sans formation musicale manifestent également de la peine à les identifier. Par contre, ces violations fines sont identifiées parfaitement par des musiciens professionnels, chez lesquels ces incongruités évoquent des réponses électroencéphalographiques sélectives rapides et intenses. Si ces réponses sélectives dérivent d'une plasticité cérébrale suite à un entraînement intensif, elles devraient s’exprimer par des composantes de potentiels évoqués (ERP) caractéristiques, dérivant d'activations cérébrales spécifiques dans des aires sensorielles, motrices et/ou cognitives, à des moments précis après la présentation de la terminaison incongrue. Nous avons pu démontrer des différences saillantes dans les réponses comportementales et électroencéphalographiques en fonction de l'expérience musicale à tous les niveaux d'analyse, et mettre en évidence, grâce à des analyses statistiques des sources présumées des ERPs, des activations cérébrales spécifiques chez les musiciens experts dans des aires temporales médiales droites, insulaires frontales et pariétales. Nous pouvons considérer ces réponses comme des adaptations cérébrales liées à un apprentissage musical explicite et intensif.
L'apprentissage implicite semble limité, et afin de progresser davantage, un entraînement explicite et intensif est nécessaire. Un tel entraînement adapte inévitablement le cerveau.
I. GENERAL INTRODUCTION………... 1
1. MUSICAL EXPERTISE ………... 2
Musical experts as a model for experience driven brain plasticity………… 2
Technical and expressive components of expert performance……….. 3
Neuroplasticity………... 4
Structural and functional brain adaptations in musicians……….. 6
2. SOME THEORY OF MUSIC……… 8
Consonance and dissonance………... 9
Musical syntax………... 10
Harmony………. 11
Rhythm……… 12
Interaction of harmony and rhythm………... 14
Musical priming………. 14
3. COGNITION AND EMOTION IN MUSIC……….. 16
Learning, memory and cognition in musical conduct……… 16
Music and emotion………. 18
4. THE PRESENT THESIS………... 21
Experimental strategy………. 21
Methodology……….. 22
Participants……… 22
Musical stimuli………... 23
Behavioural methods……….. 24
Electrical neuroimaging methods……….. 25
MMM Event Related Potentials (ERPs) ……….. 26
MMSpatio-temporal ERP analysis………... 27
MM Estimation and statistical parametric mapping of distributed sources... 28
General Hypothesis……… 29
Brief description and main results of Experiments 1-5……….. 30
Respective contributions……… 30
Experiment 1……….. 31
EEG recordings of Experiments 2-4……….. 33
Experiment 2……….. 33
Experiment 3……….. 35
Experiment 4……….. 36
Experiment 5……….. 38
II. EXPERIMENT 1……… 40
1. ABSTRACT………... 40
2. INTRODUCTION………. 40
3. METHODS……… 42
4. RESULTS……….. 47
Likert ratings……….. 47
Response times………... 50
5. DISCUSSION……… 52
Likert ratings……….. 52
Response times………... 54
6. CONCLUSION……….. 55
III. EXPERIMENT 2………... 56
1. ABSTRACT………... 56
2. INTRODUCTION……….. 57
3. METHODS……… 61
Participants………. 61
Materials………. 62
Behavioural validation of musical stimuli………. 63
Behavioural task………. 64
EEG acquisition and raw data processing……….. 65
Procedure of ERP analyses……… 65
4. RESULTS……….. 68
Behavioural results………. 68
ERP results: Waveform and topography analyses………. 69
ERP waveform analysis………. 70
Topographic dissimilarity analysis……… 71
Spatio-temporal ERP analysis………... 71
Statistical source analysis of ERPs……… 77
5. DISCUSSION……… 80
Early differences: ERP analysis………. 80
Comparisons of the early negative component found in experts with the ERAN……… 81
Early differences: Statistical analysis of putative sources………. 84
Early differences: Hemispheric asymmetry………... 87
Later differences: ERP analysis………. 87
Later differences: Statistical analysis of putative sources……….. 88
Musical Laymen………. 89
6. CONCLUSION……….……….……….……... 90
IV. EXPERIMENT 3……….……….………. 91
1. ABSTRACT………... 91
2. INTRODUCTION………. 92
3. METHODS……… 96
Participants………... 96
Materials………... 97
Behavioural Task………... 99
Stage 1……….……….……….. 100
Stage 2……….……….……….. 100
Stage 3……….……….……….. 101
4. RESULTS………...…... 103
Behavioural results………. ………..……...……... 103
Results of exploratory ERP scalp analyses……….………... 104
Results of spatio-temporal ERP analysis……….……….. 105
Results of statistical analysis of source estimations……….…….. 108
5. DISCUSSION……….……….……….. 110
6. CONCLUSION……….……….……… 113
V. EXPERIMENT 4……….……….………... 114
1. ABSTRACT……….……….………. 114
2. INTRODUCTION……….……….………... 114
3. METHODS……….……….……….. 118
Participants……….……….………... 118
Materials and procedure……….……….………... 118
EEG acquisition and raw data processing……….………. 119
Procedure of ERP analyses……….……...……...……...……... 119
Stage 1……….……….……….. 119
Stage 2……….……….……….. 120
Stage 3……….……….……….. 121
4. RESULTS……….……….……… 121
Stage 1……….……….……….. 122
Stage 2……….……….……….. 125
Stage 3……….……….……….. 127
5. DISCUSSION……….……….……….. 129
6. CONCLUSION……….……….……… 130
VI. EXPERIMENT 5……….……….………. 132
1. ABSTRACT……….……….………. 132
2. INTRODUCTION……….……….……… 132
3. METHODS……….……….……….. 135
Participants……….……….………... 135
Materials……….……….………... 136
Procedure and Behavioural Task………. 138
Normalisation of data………. 140
4. RESULTS……….. 141
5. DISCUSSION……… 145
6. CONCLUSION……….. 147
1. MUSIC PROCESSING……….……….……… 148
The developmental course of music perception……….……... 148
Musical expertise……….……….………. 149
Processing speed……….……….……….. 150
Harmonic incongruity processing……….……...……...……... 150
Metric incongruity processing……….……….……... 154
2. LIMITS OF THE CURRENT THESIS STUDY……….……...…... 155
Behavioural measures……… 155
Questionnaire……….……….……….…….. 155
Control of stimuli……….……….………. 156
Western tonal music……….……….……...……... 156
Source localisation……….……….………... 156
3. FUTURE PERSPECTIVES……….……….……... 157
REFERENCES……….……….……….……….. 160
APPENDICES……….……….……….………... 171
APPENDIX 1……….……….………... 172
APPENDIX 2……….……….………... 173
APPENDIX 3……….……….………... 174
APPENDIX 4……….……….………... 175
APPENDIX 5……….……….………... 176
APPENDIX 6……….……….………... 177
APPENDIX 7……….……….………... 178
GENERAL INTRODUCTION
Analogous to articulate language, musical activity constitutes a hierarchical organization of sequences of sounds into meaningful structured entities, requiring intricate sensory-motor coordination for its expression. All healthy individuals become experts in at least one language, but few dedicate their life to intensive and explicit music study. Therefore musical experts are an excellent model to study experience-driven cerebral neuroplasticity. However, virtually all individuals are exposed to music of their culture from birth; many receive at least basic musical instruction and some keep up musical activities throughout their lifetime as amateur instrumentalists or active listeners. This continuum of expertise levels and also the fact that virtually all human cognitive functions are involved in musical activities recently promoted musical conduct to a privileged object of research in cognitive psychology and the neurosciences.
We investigated music and auditory processing as a function of experience in adults and children, using electrical neuroimaging and behavioural approaches. In order to do so we exposed adult subjects with different expertise levels and children within different age groups to expectancy violations in expressive music and also to an auditory oddball paradigm. Our core interest was focused on functional brain adaptation in expert pianists, compared to musical laymen, in the processing of regular and incongruous expressive musical stimuli. We could demonstrate striking differences in music processing as a function of expertise at all levels of analysis.
The functional brain adaptation addressed in the current thesis study consists in neuroplasticity of learning and memory, as a consequence of enduring training.
Neuroplasticity is hypothesized to be conveyed by changes in synaptic connectivity
between brain cells within or between brain areas. This presumed neuroplasticity in expert musicians could manifest as 1) increased accuracy or sensitivity in overt judgment of musical incongruities that would reveal top down processing of an internalized musical syntax, 2) shortened latencies, increased amplitudes and idiosyncratic scalp voltage topographies of electrophysiological responses that would convey an altered filtering of incoming sensory information (bottom up) and its subsequent further processing (top down), deriving from 3) differential activations of brain areas and circuits changed by intensive and enduring training in audio-motor, memory, cognitive function and emotion related areas.
1. MUSICAL EXPERTISE
Musical experts as a model for experience driven brain plasticity
The exclusive allotment of high level musical training and its ensuing abilities confine an elite role to musical experts in the research on experience-driven cerebral neuroplasticity (Koelsch et al., 2002; Munte et al., 2002; Peretz and Zatorre, 2005;
Schlaug, 2001). Studying and gradually mastering a musical instrument requires extensive perceptual, procedural and motor learning –on an average 10.000 hours of practice precede a first stage performance of a young instrumentalist– that inevitably results in plastic reorganization of the human brain (Pascual-Leone, 2001). Various studies demonstrated that auditory and motor cerebral activations are enhanced in response to musical stimuli in professional musicians, and that sensory and motor areas are intricately coordinated in this population (Haueisen and Knosche, 2001;
James, 2004; James et al., 2008a; James et al., 2008b; Lotze et al., 2003; Schneider et
al., 2005; Zatorre et al., 2007). Perception and action are so intimately linked in an expert instrumentalist that an isolated perceptual musical input will generate execution related motor responses and vice versa.
However, musical activities can not be confined to perception and motor skills. These obvious ingredients of musical virtuosity and their interplay, like enhanced auditory- motor coordination in musicians, can hardly be disentangled from cognitive and emotional aspects of musical conduct.
We were able to demonstrate recently that plastic changes in right limbic areas (encompassing amygdala and hippocampal complex) and right insula mediate processing of subtle chord violations in professional pianists compared to laymen (James et al., 2008a). This limbic region activation might be a nexus of cognitive and affective higher order music processing.
Technical and expressive components of expert performance
Expert performance comprises two major components, a technical and an expressive one (Sloboda, 2000). The technical or "virtuoso" component requires chunking the numerous degrees of freedom (Lotze et al., 2003) in order to reduce the combinatorial explosion for all the limb and posture movements required to convert complex musical compositions into articulate sound. Supplementary degrees of freedom are required for the integration of these motor actions with high level auditory and, in the case of ensemble playing also visual perceptual feedback. Actually, motor programming in musical execution only forms part of a higher level action plan. The expressive component consists in the volitional variations of performance parameters
that manipulate cognitive and affective consequents for the listener (Sloboda, 2000).
Although these two components can be considered as independent –a technically perfect performance can lack expressiveness– they are functionally intricately linked.
Expressive and technical elements of musical performance as well as online corrections based on perceptual feedback are all conveyed by motor behaviour, and we can consider the expressive constituent the last element in the chain of degrees of freedom to be integrated. The result is a Gestalt that comprises all technical and affective components that seems so easy when a gifted and established performer executes on stage.
Neuroplasticity
Neuroplasticity or cortical plasticity is the lifelong ability of the brain to reorganize its neuronal circuits as a function of new and enduring experiences, resulting in functional and morphological remodeling of the brain. In order to learn or memorize new facts or skills, long-lasting functional changes inevitably occur in the brain representing the new knowledge. In a neuropsychological perspective, these memory traces can be considered as engrams (Dudai, 2004), biophysical or biochemical changes in the brain in response to external stimuli. According to the most current hypothesis (Martin et al., 2000; Martin and Morris, 2002), underlying brain mechanisms of neuroplasticity are adjustments in the strength of synapses between brain cells that can be operated either within the internal structure of the neurons or by an increase in the number of synapses between neurons. At the cellular level LTP and LTD, the long-term potentiation and depression of excitatory synaptic transmission,
modifications of synaptic connectivity (Malenka and Bear, 2004). However, LTP and LTD are certainly not the only means to modify neural circuit behaviour (Malenka and Bear, ibid), but that issue would go far beyond the framework of this thesis that is embedded in cognitive and not in fundamental neuroscience.
However, it remains difficult to link synaptic plasticity directly to apparently correlated behavioural responses and therefore this relationship can not be irrefutably classified as causal (Brecht and Schmitz, 2008; Martin and Morris, 2002). In that context, studying the temporal dynamics of cerebral music processing would allow a better understanding of the interplay between behavioural and cerebral parameters of music processing. If certain brain processes possess predictive value for subsequent behaviour that would be an indication of causality.
According to the website of the Vanderbilt Kennedy Center (2008), plasticity can occur in the brain under four main conditions: first, when the immature brain starts to process sensory input (developmental plasticity), second, when peripheral sensory activity is changed, for instance with altered eyesight (activity-dependant plasticity), third, when individuals' behaviour changes as a consequence of new and enduring experiences (plasticity of learning and memory) and fourth, following direct brain damage (injury-induced plasticity). All four types of plasticity are expressed by adjustments in the strength of connections, or synapses, between brain cells.
The type of plasticity addressed in the current thesis is the third type of plasticity, that of learning and memory, that manifests as the perceptual internalization of musical syntax, and is supposed to be founded on synaptic changes like all other forms of neuroplasticity.
We hypothesize that practicing a musical instrument trains the brain more intensively than sole intensive listening because of the multiple sensory and cognitive feedback loops that take place during instrumental training, which might enhance synaptic plasticity and learning. Perception then becomes kinesthetic perception and musical patterns can be represented cognitively as patterns of movement.
Structural and functional brain adaptations in musicians
Is talent decisive to achieve complete mastery in instrumental musical skill, or rigorous training? Brain studies have not been able to answer this "nature or nurture"
question in a satisfactory way because it is difficult to differentiate cause from effect (Levitin, 2006). Clear effects of musical training on children’s brain structure and cognitive development have been demonstrated in a longitudinal study comparing children learning a musical instrument to a control group matched in age, socioeconomic standard, and verbal IQ (Schlaug et al., 2005). In the sports domain, some authors consider the nature-nurture dualism no longer relevant. In their point of view each of the major interacting constraints might act in a compensatory manner on the acquisition of elite athletic performance (Davids and Baker, 2007).
Returning to musical aptitudes, the fact that violin players have larger representations of their left digits in the primary somatosensory cortex can hardly be explained by talent (Elbert et al., 1995). Statistical modelling of musical expertise (studying experts, amateurs and musical laymen) combined with voxel-by-voxel morphometric methods could establish a significant positive correlation between musician status and increase in gray matter volume in specific areas (Gaser and Schlaug, 2003). Taken
into account that amateurs took an intermediate position in the statistical modeling, this study provided strong arguments in favor of structural adaptation of the brain as a consequence of training. Structural differences between musicians and non musicians have been repeatedly demonstrated in amongst others the corpus callosum, the cerebellum, Heschl’s gyrus and primary and associative motor areas (Gaser and Schlaug, ibid; Schlaug, 2001; Schneider et al., 2002).
These structural adaptations of the brain seem intimately related to functional dissimilarities: cortical networks processing music differ between novices, amateurs and professionals while executing music related tasks (Schneider et al., 2002;
Schneider et al., 2005). For instance increase in gray matter volume in Heschl’s gyrus (structural level), evaluated by structural imaging, could be correlated with increased neurophysiological responses (functional level, evaluated by magnetoencephalography (MEG)) to auditory stimuli (Schneider et al., 2002); in this experiment amateurs' results took an intermediate position for both structural and functional differences. The same authors (Schneider et al., 2005) could demonstrate recently that left or right hemispheric dominance of MEG responses was a function of differential pitch perception (functional level) and correlated with asymmetries of specific nature in Heschl’s gyrus (structural level evaluated by magnetic resonance imaging (MRI)). Furthermore these differences in perception could be linked to the practice of different instruments of the symphony orchestra, stressing once more the influence of training. A research comparing amateur to professional violinists (Lotze et al., 2003) showed that although amateur and professional violinists relied on a similar cerebral network during left hand execution of a Mozart concerto in a fMRI scanner, cerebral activations of experts were much more focalized. This common
network comprised perisylvian and auditory areas, but also associative motor areas in the superior parietal lobe.
Together these results seem to demonstrate that brain plasticity as a function of expertise is a double edged sword. On the one hand more gray matter is available in areas that subserve musical execution and perception: specific processing areas developed; on the other hand, when a circumscribed task is performed, activations in these brain areas are concise. This probably demonstrates the reduction of degrees of freedom at a cerebral level.
Altogether, training effects on the brain seem a fact, even if predisposition can play an important role during acquisition of musical and instrumental skills. We hope that the experiments undertaken within the framework of this thesis can help disentangle more the relative contribution of predisposition and practise between skilled instrumentalists and musical laymen in cerebral music processing.
2. SOME THEORY OF MUSIC
Integrating professional musicianship1 and neuroscientific approaches, we would like to present some basic principles of musical theory that are relevant to the present thesis study. We hope that this chapter will facilitate reading this manuscript to those that are less familiar with musical theory.
Consonance and dissonance
The fundamental dichotomy between consonance and dissonance plays a key role in the generation of cognitive, aesthetic and emotional responses to music and in the organization of all tonal music in general. A consonance (Latin: consonare, "sounding together") in music is a harmony or chord considered stable and pleasant, in contrast to a dissonance considered unstable and unpleasant. Within a musical context, the consonance-dissonance issue touches the basal function of music: to satisfy the human need for accord and relaxation in balanced alternation with tension and movement (Mazzola et al., 1989). The most widespread physical definition of consonance-dissonance is by means of frequency ratios. Simple low ratios like the octave (1:2), the fifth (3:2) or the major third (5:4) are classically considered as more consonant than complex ones like for instance a minor seventh (ratio 16:9) (Pythagoras ca 600 BC; Apel, 1972). But within a musical context, within the framework of this thesis study and of all cited experiments of western tonal music, such a physical definition can not explain the qualitative culturally coded characteristics of musical chord function.
A difference should be made between consonance-dissonance within a musical context and sensory or psychoacoustic consonance-dissonance of an isolated sound.
Several psychoacoustic theories tried to explain why musical intervals with low frequency ratios are experienced as more pleasant than those with higher ratios.
Helmholtz (1863/1968) formulated the most prominent one that explains dissonance
by interaction of nearby overtones2 that causes "beats or roughness3" that our brain doesn't appreciate (Mazzola et al., 1989). Perception of sensory consonance- dissonance is invariant across cultures and largely independent of musical training (Fishman et al., 2001). Human infants perceive isolated musical consonant versus dissonant chords similarly to expert adult listeners (Zentner and Kagan, 1996).
Whereas isolated sensory consonances/dissonances can be considered a purely acoustical phenomenon, musical consonance-dissonance resides in culturally determined characteristics (Mazzola et al., 1989), and relative perception of consonance versus dissonance then varies with experience and learning. This explains why a sensory consonant chord consisting of low ratio intervals can be judged unpleasant thus so to speak dissonant, when it does not fulfill expectations within a tonal context.
Musical syntax
All types of music are guided by rules that determine how individual tones, chords and their relative durations are arranged to form meaningful musical phrases (Koelsch, 2005). Music is a multimodal medium and three principal components can be distinguished: pitch (encompassing melody, harmony and tonality), rhythm (encompassing meter, pattern, duration and tempo) and timbre (sound colour). We manipulated the following musical constituents in the context of our experiments in
2 An overtone is a natural resonance or vibration frequency of a system. Such a system can be a piano or a violin string for instance. The overall combination of the specific overtones of an instrument determines the timbre or sound colour.
3 "Beats" (basic frequency < 20 Hz) and "roughness" (basic frequency > 20 Hz) are caused by interactions in the auditory periphery between adjacent overtones of complex tones comprising a
order to create expectation violations: harmony (Experiments 1, 2 and 5), melody (Experiment 5), duration (Experiment 1) and meter (Experiment 1 and 3). We will discuss now some complex aspects of musical syntax that are pertinent within the framework of this thesis study.
Harmony
In Western tonal music, harmonic progressions follow relational syntactical rules.
Harmony refers to the procedure of musical chord construction and the rules by which one chord follows another in time. These chords are tertian4 sonorities that are constructed as stacks of thirds relative to an underlying scale, key centre or tonality.
Strong hierarchical relationships within and between chords exist, and a listener’s expectation is based on the most common sequences and composition of chords within a certain context. Rules were most concise in the classical era (1730-1820).
However, almost all popular music today is still based on classical tonality: in consequence tonality rules are quite well known by the population exposed to these stimuli, even without any explicit training (Tillmann et al., 2000). Nine month old infants of western culture already show a preference to stimuli based on the western diatonic scale (Trehub et al., 1999) and older children and adults without any formal musical education were able to identify music-syntactically irregular chords, but experts’ cerebral responses were stronger (Koelsch et al., 2005; Koelsch et al., 2002).
At the end of a musical phrase, a very specific harmonic consequent is expected.
Tonal music formulas that signify the end of a phrase involve acknowledged
4 Tertian harmony refers to western tonal harmony that is built by ascending thirds.
conventions, especially of harmonic nature, conveying a sense of completion. Such end formulas or "cadences" consist of a particular series of chords. At the very end of a musical piece, any other than an authentic cadence, comprising a fifth degree (chord build on the fifth note from the key center), followed by a first degree or tonic chord (chord build on the key center), will fail to provide full release from previously generated harmonic tensions. Furthermore this consequent chord is also expected to arrive at a particular time.
Rhythm
Rhythm is the general term for all musical time structures encompassing meter, pattern, duration and tempo. Tempo is the general speed of pulses that can be expressed in beats per minute. Duration is the relative length of a musical note, within the rhythmical context of tempo and meter, which is expressed by its notation:
crotchet, quaver and so forth. Rhythm patterns are patterns of duration, the most prominent example is probably the beginning of Beethoven's fifth symphony.
Musical meter, the main interest in the rhythmic domain in the present thesis study, consists in the presence of a regular pattern of strong and weak pulses or beats,
"periodicities", distinguishing for instance a waltz (3 pulses: strong-weak-weak) from a march (4 pulses: strongest-weak-strong-weakest), and constitutes the underlying temporal structure of pulses on which rhythm patterns can be build. A series of unstressed pulses will give an impression of incompleteness and the same holds for a series of stressed beats, because we expect an understandable relationship between accent and release. It is actually the establishment of a regularly recurring accented
beat that we are waiting for (Meyer, 1956). This underlying metrical structure allows a listener to organize incoming information and anticipate future events in a dynamic way (Hannon et al., 2004).
Fraisse (1982) already stressed in his pioneer studies on rhythm the fundamental role of a basic periodic pulse in both perception and physical activity. Basic human actions, like sucking in the newborn, walking and heart beating occur in periods within the 0.5 - 1 second range approximately. This overlaps with average natural speed of tapping, although considerable inter individual differences occur. Humans naturally synchronize their motor patterns with a regular sound sequence, which demonstrates the strong link between perception and production, but also the capacity to anticipate.
Two basic conclusions arise from basic research on the psychology of rhythm (Krumhansl, 2000). First, the human capacity of rhythm perception and reproduction rather depends on pattern detection than on the organization of individual elements.
Evidence of animal studies could demonstrate (Robin et al., 1990) that at a level as low as the auditory nerve unambiguous patterns are treated preferentially compared to ambiguous patterns. Second, rhythm perception appears intricately linked to rhythm production. Results from studies on rhythm perception systematically also apply to rhythm production, which suggests a strong motoric component in the psychological representation of rhythm (Krumhansl, 2000).
Interaction of harmony and rhythm
The majority of theoretical frameworks for understanding musical cognition have treated the perception of pitch relationships separately from the perception of metrical and rhythmic structure, implicitly assuming these to be autonomous domains (Bharucha, 1989, cited by Schmuckler and Boltz, 1994). But it seems that rather interplay between harmonic and metric elements in natural musical stimuli determines perception of completeness at musical closure. Some authors (Schmuckler and Boltz, ibid) presume a joint relationship between temporal and pitch (harmonic and melodic) structure that allows, by means of meter and rhythm, to highlight and facilitate attentional tracking, and more important to support anticipation or musical expectancy.
Hence, in order to investigate natural processing of musical rhythm or harmony, it can best be studied in a full musical environment, and not in an isolated way.
Musical priming
When expectancies are met, processing of a target is faster. This mechanism of facilitation of processing of a target by a preceding context is called priming and is a manifestation of implicit memory (Neely, 1991). The concept of musical expectancy or priming, a listener’s ability to anticipate upcoming musical events on the basis of previous experiences, is at the musical theoretical base of this thesis study (Bigand et al., 2003; Schmuckler and Boltz, 1994; Tillmann and Bigand, 2004). Musical expectations are founded upon memories of the most frequent harmonic and temporal
expectations may constitute one main root of cognitive, artistic and emotional effects in music (Meyer, 1956).
Harmonic priming is based on the learned associations between chords that differ in tonal strength (the tonic chord is the strongest; Tillmann and Bigand, 2004). The greater the tonal distance between an incoming chord and its preceding context, the greater processing costs will be, presumably because the incoming chord was not predicted and thus has a low activation level, i.e. was not “primed” by spreading activation (Patel, 2003).
However, the influence of learning on harmony processing is an issue of debate; some authors argue that acoustic short term memory may account for identification of harmony violations (Huron and Parncutt, 1993; Parncutt and Bregman, 2000, cited by Bigand et al., 2003). These conclusions were overruled by an experiment (Bigand et al., 2003) that could demonstrate that cognitive or syntactic priming prevails over sensory priming even at very fast tempi. Controlling acoustical factors, a target that was highly expected according to tonality rules was processed faster than a target that was better matched at the sensory level but that was tonally incongruous. An earlier event related potential5 (ERP) study using similar stimuli (Regnault et al., 2001) revealed differences in processing for highly expected versus less expected targets, opposing amateur musicians to non-musicians. Results showed an early effect of sensory consonance that was presumed to reflect bottom-up influences, with increased amplitude of the N16 ERP component for musicians only. Later in time, clear differences in neural processing between highly related and less related targets could be established; these differences occurred in both amateur and non musicians alike.
5 Cf. Event Related Potentials (ERPs), pp. 26
6 The N1 is a negative ERP component that occurs around 100 ms after stimulus onset.
This experiment provided strong arguments in favor of independent processing of sensory and harmonically incongruous stimuli that did not show any interaction.
Harmonic chord function is a solely auditory and music specific matter, defined within a culture; its perception and appreciation develop with experience. In contrast, the perception of musical meter is only a musically specific form of the human general capacity to synchronize attention and motor behaviour with temporally regular events (London, 2004). Human metrical skill in musical context is therefore closely related to other skilled rhythmic behaviours, like speech production and comprehension, visual and auditory tracking of moving objects and last but not least kinematic behaviours like dancing or sports (London, ibid). In consequence musical laymen with experience in alternative skilled rhythmic behaviours may apply this knowledge in the musical domain. Nevertheless, specific rules that govern musically specific rhythm are culturally determined and prone to learning.
In this thesis study we focused on cognitive or syntactic top-down processes that imply deep structures of the musical material.
3. COGNITION AND EMOTION IN MUSIC
Learning, memory and cognition in musical conduct
Although it is supposed that musical tonal contexts are maintained in brain regions that integrate sensory, cognitive and affective functions (Janata et al., 2002a), it is not clear which of these functions are shaped by musical training and what the time- course of these experience-dependent responses in the brain is. Knowing the time
course would allow a better understanding of the interplay between these different parameters of music processing.
The auditory system is dynamic. The processing of acoustic stimuli in auditory and associative areas, and thus the responses of neurons, are affected by attention and learning. Early animal studies using classical conditioning paradigms could demonstrate that neuronal responses in the auditory system to initially neutral sounds could be modulated in association with reward or punishment (Weinberger and Diamond, 1987) The actual tuning of neurons (frequency of receptive fields) in the primary auditory cortex can be altered by learning (Weinberger et al., 1993). These bottom-up or filtering processes are thus highly plastic.
Brain mechanisms underlying music perception logically starts in the auditory system, but by no means ends there (Weinberger, 1999). Music perception comprises the perceptual organization of patterns of pitch in time. Logically, this process sets off in the auditory system, and evidence of animal studies could demonstrate (Robin et al., 1990) that at a level as low as the auditory nerve unambiguous patterns are treated preferentially compared to ambiguous patterns. However, as music perception and production comprise a scope of sensory, perceptual, cognitive, mnemonic, aesthetic and last but not least emotional processes, brain substrates must transcend central auditory or perceptual processing. At this associative level the distinction between perceptual, cognitive and emotional levels of processing then becomes less clear.
Higher order pitch processing of complex musical stimuli for instance has been demonstrated to take place outside the auditory cortex, namely in limbic areas and also in the insular cortex. Patients who underwent right sided amygdalahippocampectomy performed poorly for higher order pitch discriminations
and memory for tone sequences in a musical aptitude test (Seashore Test; Wieser, 2003). Recordings from implanted electrodes in the mesial temporal lobe of an epileptic patient revealed that left hippocampus responses were modulated by traditional dissonances and right hippocampus by higher order tonality aspects of auditory stimuli (Wieser and Mazzola, 1986). When healthy experienced musical listeners were exposed to ongoing changes in tonality (Janata et al., 2002a) hippocampal activations were also demonstrated. Furthermore right insula activations in response to subtle harmonic incongruities could be demonstrated with functional imaging (Tillmann et al., 2006).
We hypothesize that synaptic plasticity in musical experts in limbic and insular structures (James et al., 2008a) reflects the formation of enhanced higher order pitch and tonality hence cognitive processing and of a highly specific auditory memory for musical syntax. Compelling evidence for such experience-dependent plasticity in the neocortex and hippocampus exists (Brecht and Schmitz, 2008; Martin and Morris, 2002). We speculate that because the limbic system and insula also are preferential sites of emotional processing, these brain areas may constitute a nexus of cognitivo- emotional treatment of musical stimuli.
Music and emotion
Musical activities are omnipresent in human culture and can be traced down to the Neanderthal era (Wong, 1997); hence, although they have no direct survival value, they might be a fruit of evolution (Huron, 2001). In an evolutionary perspective the adaptive function of music is generally claimed to be emotional communication and
regulation (Hauser and McDermott, 2003). Although the connection between music and emotion seems obvious, it is hard to provide any direct scientific proof. Some objective data have been collected though. Before discussing these, a difference should be made between intrinsic musical emotion, elicited by musical components like consonance and dissonance, complexity of harmony or rhythmic articulation, versus the recognition or induction of "basic" emotions (Oatley and Johnson-Laird, 1987) like sadness or fear within a musical material. In our point of view, the first approach is by far more interesting, because the musical acoustical environment constitutes a world of its own. Moreover, it has been suggested (Scherer, 2004) that basic emotions are too poor to do justice to our music listening experiences.
Returning to objective arguments in favour of emotional responses to music, first, intrinsic features of musical structure have been shown to correlate with physiological measures of emotion, like respiration, skin conductance and heart rate (Blood and Zatorre, 2001; Gomez and Danuser, 2007). Unexpected harmonies in Bach chorales evoked an increase of subjective emotional experience that coincided with enhanced electrodermal activity (an indicator of emotional arousal) and elicitation of typical ERP responses to harmonically unexpected events (Steinbeis et al., 2005). When classical categories of "basic" emotions were matched with existing musical pieces, emotion-specific electrophysiological changes occurred. However, these specific changes only partially agreed with those for corresponding emotions outside the musical environment. This demonstrates that emotions within and without the musical environment are not the same (Krumhansl, 1997). Second, functional cerebral imaging could demonstrate that musical “chills”, evoked when individuals were exposed to explicit passages in favourite pieces, elicited activation in the insula and midbrain "reward centers" (Blood and Zatorre, 2001). Third, a comparison between
brain correlates of pleasant and unpleasant states, induced by exposure to respectively classical masterpieces versus an atonal and athematic contemporary piece, showed that unpleasant musical emotions resulted in increased activation in right paralimbic and right frontopolar areas, associated with emotional processing (Flores-Gutierrez et al., 2007). Such a response might not occur in experienced listeners for an atonal idiom. Finally, clinical benefits of music therapy provide some indirect measures:
immune and endocrine factors significantly changed for the better in a group of patients exposed to Bach's Magnificat (le Roux et al., 2007), and an extensive review on musical therapy in hospital setting confirms the potential of this medium to improve hospital experiences of patients (Richards et al., 2007).
All these data do indicate that music connects mind and body, but the affective nature of this connection can not be confirmed irrefutably.
In Emotion and Meaning in Music the musicologist and philosopher Meyer (1956) postulates that frustration of expectation is the root of affective and aesthetic responses to music: affect is aroused when an expectation, activated by a musical stimulus, is temporarily inhibited or permanently blocked. Note that here we are clearly discussing intrinsic musical emotion here. In Meyer's (ibid) point of view, musical styles are historically evolving systems of expectations: gradually the deviants become normative within the style and provide the basis for further deviation.
However, as stated above, music is a multimodal medium and we may wonder which elements play leader roles in establishing musical expectations. Are some musical elements more prone to evoke emotional responses than others in case their
dominated by features of pitch, more specifically by pitch simultaneity or harmony (Blood et al., 1999; Flores-Gutierrez et al., 2007; James et al., 2008a, Steinbeis et al., 2005; Steinbeis et al., 2006), but sufficient data on temporal pattern perception and its neurological underpinnings lack.
4. THE PRESENT THESIS Experimental strategy
We investigated behavioural responses as well as stages of cerebral processing and associated cortical networks implicated in the detection of musical expectation violations, comparing professional musicians to non professionals (Experiments 1, 2 and 3), and also in primary school children of different ages (Experiment 5).
Syntactical incongruities of either harmony or rhythm were applied to the endings of expressive musical pieces and children's songs, composed for our purposes. A definite choice of stimuli and groups for Experiments 2 and 3 was made according to the results of a behavioural pilot study (Experiment 1), that aimed to determine the right degree of violation in order to optimally separate experimental groups by means of their behavioural responses. The resulting musical material was used in Experiments 2 and 3, the core studies of this thesis, in order to evaluate experience driven brain plasticity in the processing of subtle syntactic incongruities of harmony (Experiment 2) and of meter (Experiment 3). In order to investigate the relative contribution of central auditory processing to these cognitive processes all subjects participating in Experiments 2 and 3 were exposed to an auditory oddball paradigm (Experiment 4).
Parallel to Experiments 1-4 we investigated possible developmental changes in the
processing of harmonic incongruities in primary school children (Experiment 5). In this behavioural study we exposed participants to two levels of harmonic incongruity at musical closure: subtle (syntactic), similar to the incongruities in Experiment 2, and coarse (sensory). We applied these incongruities to simple melodies (children's songs) and also to a subset of the musical pieces used in Experiments 2 and 3.
In all experiments, excepted Experiment 4, pieces ending in an incongruent way were presented interspersed with regular versions of the musical material in pseudo randomized order. Subjects then appraised relative satisfaction provided by these endings.
Methodology Participants
For Experiment 1 we recruited adult participants within 4 expertise profiles (cf. Brief description and main results of Experiments 1-5, pp. 32).
For Experiment 2-4, in order to maximise expertise and homogeneity and also because of neurophysiological aspects, young adult male right-handed professional pianists were recruited for the expert group. Sex is known to influence neurophysiological responses (Ortigue et al., 2004; Ortigue et al., 2005), and specifically so for music processing (Koelsch et al., 2003b). All musical laymen were also men, of similar age and education level.
For Experiment 5 randomly assigned primary school children from 6 to 12 years of
Musical stimuli
Part of the originality of this thesis study resides in the choice of the musical material and the manipulations applied to these stimuli. Hitherto most research on violation of syntax has been done on chord sequences (Bigand et al., 2003; Koelsch et al., 2001;
Koelsch et al., 2007; Poulin-Charronnat et al., 2006; Regnault et al., 2001; Tillmann and Bigand, 2004; Tillmann et al., 2003; Tillmann et al., 2006). Inspired by these studies, we opted for genuine musical stimuli, because we hypothesized that offering subjects a rich and expressive musical context would evoke more natural and complete responses, and specifically would allow stronger cognitive and also affective reactions to occur in response to expectancy violations. In order to rule out the role of memory traces on musical priming, the complex musical material Experiments 1, 2, 3 and 5) was composed specifically for the occasion by the Dutch composer Nicolaas Ravenstijn7, according to our instructions. These 30 compositions were expressive polyphonic piano pieces that varied in character and in length (13.1 ± 3.8 s), thus building up a strong and realistic musical expectancy towards the terminal chord (for a more extensive description of the stimuli we refer to Experiments 2 and 3 pp. 62 and 97 respectively). Based on the results of Experiment 1 that served to validate and investigate stimuli and experimental conditions, we chose as manipulations for Experiments 2, 3 and 5 subtle syntactical incongruity of harmony and meter. These incongruities were applied to the terminal chords of the pieces, where musical expectancy is most concise. The harmonic incongruities consisted in deceptive cadences, and terminal chords were closely related consonant chords of different degrees (fourth and sixth degrees) instead of the expected tonic chord (first degree). The metric incongruities advanced the solution with one beat or pulse,
7 Nicolaas Ravenstijn: [email protected]/ www.filiuscorvi.com
therefore breaking the rule of ending on a strong accentuation (at the first beat of the measure), but still stayed within the meter. Both violations can be categorized as syntactical or cognitive and not sensory or intrinsically unpleasant (Bigand et al., 2003), and were thus only incongruous within the musical grammatical context (exemplary soundfiles can be found on the enclosed CD-ROM). We chose such refined anomalies of musical syntax in order to provoke distinct responses as a function of musical experience. For Experiment 5 simple children's tunes inspired on existing songs were created. In this study 2 different levels of incongruities were used in order to investigate the developmental trajectory of the processing of musical syntax. Therefore, in addition to the syntactical or sensory incongruities, more salient sensory or coarse violations were also applied to the stimuli.
Behavioural methods
In all experiments except for the auditory oddball that did not require an overt response, we asked our subjects to provide us with a judgment of satisfaction of the terminal chord. In Experiment 1 for which we solely collected behavioural responses, we used a 5 point Likert scale, from absolutely not satisfactory (1) to completely satisfactory (5). For Experiments 2 and 3 that used electrical neuroimaging methods, we opted for a binary response, in order to reduce motor involvement to a minimum, and subjects responded either "yes" for satisfactory endings or "no" for non- satisfactory endings. For Experiments 1-3, responses were given via a response box.
For Experiment 5 with a children's population and behavioural responses we asked participants to draw a crossbar on a continuous vertical line of 10 cm with a sad and a
Electrical neuroimaging methods
How to study neuronal mechanisms subserving music processing? The musical stimulus is carried by time and therefore should ideally be investigated by methods capable of analysing the time variable in human responses to music with great precision. However, we also wanted to investigate neuronal sources and their differential implication as a consequence of intensive training as a function of time.
We therefore chose to use electroencephalography (EEG) that, with a temporal solution down to the sub-millisecond range, allows to study sequences of cognitive brain processing in real time (Michel et al., 2001). Furthermore, although with a relative coarse spatial resolution, distributed inverse solutions are a reliable way of exploring neuronal correlates of electrical brain states (De Santis et al., 2007; Murray et al., 2006; Pourtois et al., 2005). Although the spatial resolution of distributed inverse solutions is still a matter of debate, it might actually closely approach that of standard (spatially smoothed) fMRI results (Michel et al., 2004a). The electrical neuroimaging method we chose to analyse our high density EEG data with consisted of three main procedures:
1) Extraction of Event Related Potentials from the EEG signal.
2) Spatio temporal ERP analysis: determination of functionally relevant time periods or "microstates" in the ongoing electric activity at the scalp level.
3) Estimation and statistical parametric mapping of distributed sources: localization of the putative sources in the brain that generate the activities recorded on the scalp and statistical comparison of these putative sources between groups or experimental conditions.
Event Related Potentials (ERPs)
EEG allows recording voltage fluctuations at the scalp that reflect the post-synaptic activity of large neuron populations of cortical pyramidal cells. When a large number of epochs time-locked to a stimulus are averaged, the information directly related to the processing of the events can be extracted. This process of signal averaging is necessary in order to extract the "signal" (the time locked ERP) from the "noise" (the background EEG; Britz, 2006; Rugg and Coles, 1995). The resulting ERP is any stereotyped electrophysiological response to an internal or external stimulus.
Unfortunately no clear consensus exists for the nomenclature and definition of components. In combination with order of appearance polarity can define a component (such as N1, P2: first negative and second positive peak) or peak latency (such as P300, N400). Some components are named according to their functional significance (such as MMN for mismatch negativity) or topographic distribution (e.g.
ERAN for early right-anterior negativity; Britz, 2006).
Within the framework of this thesis, one music specific ERP component is of particular interest: the ERAN. This component (Koelsch et al., 2001) has been elicited in response to harmonically syntactically irregular terminal chords in musicians and non musicians. The ERAN, with a peak latency of ~ 200 ms, is characterized by bilateral negative potentials at frontal electrodes and a right hemisphere dominance that becomes weaker with more subtle irregularities, and stronger with musical expertise (Koelsch et al., 2007; Koelsch et al., 2002). In sum this component seems to express the degree of harmonic violation, that can vary intrinsically, but also as a function of experience.
In order to allow comparison to well known ERP components we applied classical waveform analyses to our data. Repeated measures ANOVAs were performed on the variables mean amplitude (in μV), peak amplitude (in μV) and time point of peak amplitude (in ms) within pertinent time windows.
Spatio temporal ERP analysis
Determination of functionally relevant time periods was operated by means of a spatio-temporal pattern procedure applied to the ERPs. This analysis, known as microstate segmentation (Michel et al., 2004a; Michel et al., 1999; Murray et al., 2006; Pascual-Marqui et al., 1995; Thierry et al., 2007), is founded on the phenomenon that the topography of the electric field at the scalp does not vary randomly as a function of time, but rather remains in a stable configuration for brief time periods or components: microstates of information processing. These periods of stable topography, both within and between conditions or groups, are identified in the data set with a k-means cluster analysis, first at group level (hypothesis generation) and then at individual level (fitting and statistical confirmation). This procedure possesses multiple assets. In the first place the information of all electrodes is used, and analysed in a global way, in contrast to classical ERP analyses, that often concentrate on a very restricted number of electrodes. A decisive advantage of this analysis is that a distinction can be made between latency and topography differences.
A difference in latency could yield differences in a topographic dissimilarity analysis across time that would thus not be due to idiosyncratic topographies for a particular condition or group (Michel et al., 2004a). By physical laws, only when potential configuration differences exist changes in the distribution of active generators in the
brain during this period can be presumed (Vaughan, 1982). Therefore we only estimated putative neuronal sources over periods for which topography differences existed on the scalp level, that were not due to latency differences. A final asset of this technique is that it allows multi-factorial comparisons across groups (Pascual-Marqui et al., 1995), highly pertinent in our study with different experimental conditions and 2 experimental groups (Experiments 2, 3 and 4). Recent detailed description and discussion of this topographic ERP analysis method can be found in Murray et al.
(2008) and Pourtois et al. (2008).
Estimation and statistical parametric mapping of distributed sources
Because we did not have clear a priori hypotheses about neuronal sources we opted for distributed inverse solutions that are capable of dealing with multiple simultaneously active sources (Grave de Peralta Menendez et al., 2001). It should be kept in mind though that the calculated sources are a result of modeling and not of direct recording of these sources (Michel et al., 2001). We used statistical parametric mapping (SPM), comparing by means of t-tests the two groups for putative sources;
these statistical analyses can prevent non-systematic noise-related localization errors (Lantz et al., 1997; Zumsteg et al., 2005). Studies using simultaneous recordings of intracranial and scalp EEG have shown that even profound medial temporal activity can be reliably retrieved from scalp EEG, using comparable distributed source reconstruction techniques as in the current Experiments 2 and 3 (Lantz et al., 1997;
Zumsteg et al., 2005). Since scalp EEG and MEG recordings both suffer from the fact that the inverse problem is ill-posed, we chose to apply conservative statistical
Altogether these methods seemed to offer an optimal way of dealing with music and auditory perception, comparing experts to laymen, as we did in Experiments 2 and 3 and 4. To our knowledge, these methods have never been applied to analyse music perception.
General Hypothesis
In western tonal music, musical phrases end with a typical chord sequence that can be concluded only with a tonic chord. This consequent chord is also expected to arrive at a particular time. Deviance from these expectations may evoke increased cognitive processing and also frustration. As such expectation is a consequence of musical experience, cognitive and affective processing of incongruities of musical grammar might be a function of expertise and development. Although obvious violations of standard closure in musical pieces in classical style may be easily detected by non professional adults and primary school children, more subtle syntactic incongruities will be distinctively apprehended by musical experts, who have incorporated the rule system more extensively due to intensive training. Intensive musical training inevitably results in functional adaptation of the brain. In consequence a subtle syntactic incongruity of standard closure, either harmonic or rhythmic, will evoke a profound and rapid response in an expert, whereas an auditor lacking musical training may barely detect it. We therefore expected behavioural and cerebral responses to incongruous musical stimuli to alter with increased expertise. On the behavioural level we predicted increased accuracy and shorter response times as a function of expertise. At a cerebral level we anticipated shorter latencies and increased amplitudes of ERPs, idiosyncratic functional microstates and distinct neuronal
correlates for experts in areas associated with motor, sensory, cognitive and affective processing.
Brief description and main results of Experiments 1-5 Respective contributions
For all experiments Clara James' contribution consisted in execution of all experiments and all analyses, as well as the writing and editing of all articles/manuscripts, but she has been substantially and generously advised and coached by thesis director Claude-Alain Hauert (general adviser, corrections/suggestions for all articles/manuscripts), by thesis committee member Christoph Michel (guidance for all the methodological aspects of the EEG analyses, substantial contribution in the form of corrections and suggestions for the articles of Experiments 2 and 3 plus general advices), by Juliane Britz, scientific assistant of Christoph Michel (substantial contribution in the form of corrections and suggestions for the article of Experiment 2 and 4, and also some methodological help), by thesis committee member Patrik Vuilleumier (substantial contribution in the form of corrections and suggestions for the articles of Experiment 2 and 3, plus general advices), and finally by thesis committee member Emmanuel Bigand who contributed substantially in the very beginning of the thesis project with important advices on the nature of the musical incongruities.
In the following sections, we present each experiment in a summarized way. For details, the reader can refer to chapters II to VI.
Experiment 1
Judgment of musical expectation violation dissociates professional pianists from amateurs, music lovers and laymen. Manuscript in preparation.
Hypotheses
o Expectancy violations will be identified more accurately and more rapidly with superior expertise.
o The combination of harmonic and rhythmic expectancy violations will facilitate their identification especially for lower level expertise groups.
o We expect incongruities to yield longer response times.
Description
This preliminary experiment served to select groups and validate stimuli for 2 consecutive electrical neuroimaging studies (Experiments 2 and 3). We aimed to determine an appropriate degree of violation in order to optimally separate experimental groups by means of their behavioural responses. The ultimate goal of this thesis was to investigate functional changes in cerebral processing as a function of musical expertise. When behavioural responses differ, this increases the probability to find cerebral differences and will also facilitate interpreting them. The stimuli consisted of 14 expressive piano compositions in classical style, composed for this thesis project. Harmonic incongruity and two different kinds of rhythmical
incongruities were applied to the endings of these musical pieces; additionally the possible combinations of these incongruities (incongruous both for rhythm and harmony) were used, resulting in 6 versions of these pieces. These 6 versions, defining 6 experimental conditions, were respectively: regular, harmonically incongruous, rhythmically incongruous 1, rhythmically incongruous 2, harmonically and rhythmically incongruous 1 and harmonically and rhythmically incongruous 2.
The 6 versions of the 14 different pieces were presented to 32 participants that were recruited within 4 different experimental groups: musical laymen (n=8), music lovers without practical experience (n=7), amateur pianists (n=9) and expert pianists (n=8).
The participants were requested to rate the endings of the musical stimuli on a 5 point Likert scale varying between completely satisfactory (5) and absolutely not satisfactory (1).
Main results
Both musician groups were superior in identifying regular endings as compared to music lovers and laymen. In contrast for all incongruous conditions ratings of amateurs, music lovers and laymen did not differ. Professionals were highly superior for identification of harmonic incongruity. Surprisingly professionals rated rhythmic incongruities as quite satisfactory. The 3 non professional groups rated these rhythmical anomalies as less satisfactory than the professionals. Facilitation of identification for double incongruities (harmony and rhythm) only occurred in the 3 non professional groups. On the basis of the conclusions we drew from this experience the definitive nature of stimuli and groups for experiences 2 and 3 was determined.
EEG recordings of Experiments 2-4
Data for Experiments 2, 3 and 4 were collected in one session that lasted approximately 4 hours per participant, with an approximate total of 120 minutes of pure EEG recording.
Experiment 2
James, C.E., Britz, J., Vuilleumier, P., Hauert, C.-A., Michel, C.M., 2008a.
Plasticity in right limbic structures mediates harmony incongruity processing in musical experts. Article under review.
Hypotheses
o Subtle harmonic incongruities of standard closure will be recognized at ceiling by professional pianists, whereas musical laymen may hardly detect them.
o We expect an ERAN-like (Koelsch et al., 2001; Koelsch et al., 2007; Koelsch et al., 2002) ERP component in response to the syntactically incongruous stimuli, with stronger amplitude and shorter latencies in experts.
o Distinct neuronal substrates for incongruity processing in experts may be localized in areas associated with motor, sensory, cognitive and affective processing.
