Production of Stereotyped and Exploratory Vocal
Behavior in Songbirds
by
Galen Lynch
Submitted to the Department of Brain and Cognitive Sciences
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in Neuroscience
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
February 2020
© Massachusetts Institute of Technology 2020. All rights reserved.
Author . . . .
Department of Brain and Cognitive Sciences
September 06, 2019
Certified by . . . .
Michale S. Fee
Glenn V. and Phyllis F. Dorflinger Professor of Neuroscience
Thesis Supervisor
Accepted by . . . .
Rebecca Saxe
John W. Jarve (1978) Professor of Brain and Cognitive Sciences
Graduate Officer, Brain and Cognitive Sciences
of Stereotyped and Exploratory Vocal Behavior in Songbirds
by
Galen Lynch
Submitted to the Department of Brain and Cognitive Sciences on September 06, 2019, in partial fulfillment of the
requirements for the degree of Doctor of Philosophy in Neuroscience
Abstract
Whether it is speaking to one another, or nailing a tennis serve, humans can perform an incredible range of behaviors, most of which are learned. How do we and other animals learn complicated sequential behaviors, and once they are learned how are they executed? This thesis is an investigation into the neural basis of the two modes of behavior that occur at the beginning and end of learning a motor skill: the initially highly variable exploratory behavior, and the ultimately stereotyped skilled performance. To understand the start and end points of learned motor behaviors, I present two studies, each on the premotor activity of ensembles of neurons that underlie song production in zebra finches.
Executing learned motor behaviors requires animals to produce precisely timed motor sequences. While cortical motor regions traditionally have been viewed as encoding features of motor gestures (Georgopoulos et al., 1982), more recent stud-ies have suggested that motor regions may have intrinsic dynamics to pattern the production of motor gestures (Churchland et al., 2012). A similar debate has arisen in songbirds. Adult birdsong requires the premotor nucleus HVC (used as a proper noun), in which projection neurons burst sparsely at stereotyped times in the song. It has been hypothesized that projection neuron bursts, as a population, form a continuous sequence, while a different model of HVC function proposes that HVC activity is tightly organized around motor gestures. Using a large dataset of HVC neurons recorded in singing birds, we test several predictions of these models. We find that projection neuron bursts in adult birds are continuously and nearly uniformly distributed throughout song. However, we also find that firing rates exhibit significant 10 Hz rhythmicity locked to song syllables, peaking prior to syllable onsets and sup-pressed prior to offsets—a pattern that predominates activity in HVC during early stages of vocal learning.
The acquisition of many learned behaviors begins with a phase of highly variable exploration. This variability plays a crucial role in reinforcement learning and other models of learning. In songbirds, the premotor brain region lateral magnocellular
nucleus of the anterior nidopallium (LMAN) actively generates behavioral variability
variability? Theoretical studies propose that interconnected neurons within LMAN exhibit chaotic dynamics, thereby producing seemingly random patterns of activity. One such model posits that LMAN is a balanced excitatory-inhibitory network, and predicts that pairs of neurons in LMAN would have uncorrelated activity. Another model posits that LMAN may act as an excitable media producing locally propagating waves of activity, and predicts that all nearby pairs of neurons would be highly correlated. To test these models and to understand how LMAN actively generates behavioral variability, we built a miniature lightweight microdrive to simultaneously record from multiple neurons, as well as a lightweight endoscope to perform functional calcium imaging of ensembles of LMAN neurons. With these new technologies, we observed the simultaneous activity of pairs of single units in singing juvenile and adult birds. We find that most pairs of neurons with small separation (<250 µm) are completely uncorrelated, which is incompatible with the wave model. However, a small subset of pairs have strikingly large correlations, with correlation coefficients of up to 0.81. Intriguingly, these correlated pairs of neurons can be separated by up to 400 µm. The existence of such highly correlated neurons within LMAN is inconsistent with LMAN being a simple balanced excitatory-inhibitory network with uniformly random connectivity. These results suggest that new models of variability generation are required to explain how LMAN generates exploratory behavioral variability.
Thesis Supervisor: Michale S. Fee
Title: Glenn V. and Phyllis F. Dorflinger Professor of Neuroscience
I distinctly remember when I just knew that I had to learn from Michale Fee, who has been my supervisor and mentor for the last eight years. He was giving a quick presentation of his research to me and the other first-year grad students, and over the course of just about 30 minutes he completely changed my understanding of what was possible in neuroscience. His careful dissection of a complex task—song learning—into small mechanistic pieces that can be individually recorded, inactivated, and cooled with bespoke devices made for an incredibly elegant and compelling account of how the whole bird worked. Only after joining his lab did I realize how much effort goes into each and every one of these discoveries, how many models are drawn on the white board only to be erased, or how many devices were carefully crafted but were just a little too clever to work. Through it all, he remains focused on the core scientific questions, and exactly which data and experiments would nail the question, and never settles for less. I have learned an immense amount from watching him throw all his intellectual force against a problem day after day, until he comes to the elegant and
seemingly effortless solution that goes into his presentations. It was an honor to work
with him and learn just a fraction of what makes him the scientist that he is, and I will miss working with him and the other amazing people that are drawn to him.
I am deeply grateful to my committee members, who have shaped my thinking in so many ways. Matt Wilson and Jim DiCarlo were both kind enough to let me rotate through their labs, which has left a lasting mark on my scientific thinking. Their lab cultures and lab members have also informed the work presented here.
Matt Wilson’s lab has been a foundry for electrophysiological techniques and electronics for as long as I’ve been at MIT. When I was rotating through his lab, Josh Siegle and Jakob Voigts were making next generation ephys tools out of his lab, tools that I would use throughout my PhD to record much of the data presented here. Yet in the very next room, others in his lab were using the ephys rig that Matt designed during his postdoc which, despite seeming to blur the line between space shuttle and Tron, nevertheless remained cutting edge so many years after he designed it: a testament to Matt’s technical prowess. Most of the devices that I made during my time at MIT, and all of the devices in chapter 3 and chapter 4, were designed in consultation with members of his lab; in fact a good fraction of those devices were soldered in his toaster oven. His advice during committee meetings, teachings in class, and conversations both after seminars and at departmental events has been invaluable to me, and has reminded me to not shy away from linking systems neuroscience to cognition in a meaningful way. His work on sequences in learning and memory also informed how we tested for sequential activity in chapter 2.
Jim DiCarlo’s lab meetings were unlike any that I’ve been to before or since rotating through his lab: the focus on rigorously defining what the question is, and really giving definitions the respect that they are due, produced a clarity of discourse that is the foundation for the extremely clear science that he produces. After deciding on a scientific question, or seemingly any other pursuit in life, he pursues it with the dedication and stamina of the marathon runner that he is. I am grateful that he let
to his lab paintball retreat, giving me a chance to absorb his lab culture and scientific mindset. That both of the projects presented here are quantitative tests of models using ephys data is no accident, and was in part inspired by his dedication to do the very same with models of vision.
I would like to thank Bence Ölveczky for agreeing to be my outside committee member. Much of the work presented here on LMAN follows directly from that of his own, both while he was working with Michale and afterwards. His analysis techniques used to study HVC, and his findings in RA were also integral to the analyses used in chapter 2. His ability to translate lessons learned from the bird into mammalian systems has also been personally inspirational to me.
I have been extremely fortunate to work with amazing collaborators, without whom none of this work would have been possible. Tatsuo Okubo was my co-first author for the work presented chapter 2, and single-handedly recorded 4/5 of the birds in that study, increasing the largest such dataset by manyfold. Without him, there would have been no study. As a senior grad student in Michale’s lab when I started, he was a role model to me in many ways, and his excellence in techniques and scientific thinking has been inspirational. Harbi Sohal has been an excellent colleague and friend, and was the driving force behind the flexible probes in chapter 3. Ed Boyden has also played a big role in making these flexible probes possible. Joe Scherrer and I have been working together to both make the endoscope in chapter 4 as well as image activity in LMAN. Even though I’m supposed to be his senior, he is constantly teaching me new things on a wide variety of topics. Natasha Denisenko is one of the main reasons why there is any functional calcium imaging in my thesis, and taught me much of what I know of histology and anatomical techniques. Alexander Hanuschkin and Richard Hahnloser poured many, many hours into a model that none of us believed, and made the model testing in chapter 2 possible. While Michael Stetner and I never officially collaborated, he was a sounding board and circuit proof reader for every project presented here. Adrienne Fairhall and Alison Duffy have both been helpful in formulating our thoughts on the activity in LMAN. Finally, far from being confined to an advisory role, Michale Fee was in the trenches with me on most aspects of every project.
It has been an absolute pleasure and privilege to work along side the other members of Michale Fee’s lab. In alphabetical order, I had the good fortune to overlap with: Andrew Bahle, Maya Bronfeld, Tim Currier, Husain Danish, Natasha Denisenko, Shelbi Ferber, Jesse Goldberg, Shijie Gu, Michael Happ, Anton Konovchenko, Joergen Kornfeld, Emily Mackevicius, Yael Madelblat-Cerf, Collyn Messier, Kail Miller, Aditya Nair, Anusha Naryan, Nader Nikbakht, Tatsuo Okubo, Margo O’Leary, Dan Rubin, Joe Scherrer, and Michael Stetner. I could not have asked for better colleagues. I am particularly thankful to Jesse Goldberg and Michael Stetner for teaching me so many techniques when I first joined the lab; Tatsuo Okubo, Natasha Denisekno, and Joe Scherrer for being wonderful collaborators; Emily Mackevicius for countless stimulating conversations and camaraderie; Joergen Kornfeld for reading more of this thesis than is reasonably fair; Andrew Bahle for being a great scientific peer; and Nader Nikbakht for his unfailing support. I have also benefited from interactions with
devices in this thesis, and I will be forever grateful to Mike Long for delaying his publication to allow us to publish back-to-back.
I have met many great people while at MIT, too numerous to list here. I have directly benefited from the gadget and software wizardry of Jon Newman, Jakob Voigts, Jie (Jack) Zhang, and Josh Siegle. I was a proud member of the Persian History Club, whose members showed me by example that it is possible to both be a good scientist and also have wonky time-consuming hobbies. These members included Arash Afraz, Stefano Anzellotti, Nima Dehghani, Dan Yamins, Leyla Hormozi, Noa Corcoran-Tadd, Emily Mackevicius, and Vladislava Chalei.
Many of the people listed above I also count as friends, but I would like to thank my four previous house-mates who grew to be close friends and helped to make grad school such a fun place: Praneeth Namburi, Zenna Tavares, Julian Jara-Ettinger, and Max Siegle. Huge thanks to Asher Bartch, who has been like a brother to me over these many years. I would also like to thank in alphabetical order Alec Chapman, George Chen, Andrew Dane, Omer Durak, Elliott Ferguson, Ylaine Gerardin, Allena Goren, Phillip Isola, Elias Issa, Alex Kell, Shaiyan Keshvari, Dorit Kliemann, Bhavna Lal, Brian Leyde, Raunaq Malhotra, Jon Malmaud, Steve McQuillan, Wiktor Mlynarski, Edward Nieh, Rick Pampuro, Hilary Richardson, Elaine Short, Taylor Stevenson, James Traer, Valerie Wallace, Neal Waters, Sarah Yun, and many others for their friendship over the years.
I want to thank my family for their support through my entire life, but particularly during grad school: My parents Jonathan Lynch, Judith Lynch, and Kathleen Brown; my siblings Emily Lynch, Andy Evensen, and Lori Evensen. Lastly, I want to thank Steffi Luk, for helping me in more ways than I can put into words: none of this would have been possible without you.
Thank you all.
List of Figures 13
List of Tables 17
Acronyms 18
1 Introduction 23
1.1 Acquisition and Execution of Complex Motor Behaviors . . . 24
1.1.1 The Neural Basis for Exploratory Behavior . . . 26
1.1.2 The Neural Basis for Execution of Learned Motor Behaviors . 27 1.2 The Songbird as a Model Organism for Learned Behaviors . . . 28
1.3 The Song System: a Network of Nuclei for Learning and Producing Song 33 1.4 The Role of HVC in Song Production and Learning . . . 38
1.4.1 Models of HVC activity . . . 40
1.5 The Role of LMAN in Song Production and Learning . . . 42
1.5.1 Models of LMAN Activity . . . 45
1.6 Summarized Results and Attributions . . . 47
2 Rhythmic Continuous-Time Coding in the Songbird Analog of Vocal Motor Cortex 49 2.1 Results . . . 50
2.1.1 HVC Bursting Produces Continuous Coverage of Time in the
Song . . . 50
2.1.2 Quantification of HVC Clustering . . . 55
2.1.3 Coordination of HVC Projection Neuron and Interneuron Activity 59 2.1.4 Quantification of Alignment of Bursts and Minima to Temporal Features of Song . . . 62
2.1.5 Covariation of HVC Spike Timing and Song Timing . . . 66
2.1.6 Covariation of HVC Activity with Song Spectral Features . . . 70
2.1.7 HVC Firing Rates Exhibit Slow Syllable-Related Modulations 75 2.1.8 HVC Activity in Juvenile Birds . . . 80
2.2 Discussion . . . 86
2.3 Experimental Procedures . . . 91
2.3.1 Subjects . . . 91
2.3.2 Song Recordings . . . 93
2.3.3 Neural Recordings . . . 93
2.3.4 Calculation of Burst Times . . . 94
2.3.5 Interneuron Firing Rates and Minima . . . 95
2.3.6 Gesture Trajectory Extrema (GTE) Analysis . . . 95
2.3.7 GTE Estimation . . . 98
2.3.8 Generation of Surrogate Datasets with Monte Carlo Sampling 99 2.3.9 Statistical Testing . . . 99
2.3.10 Coverage Analysis . . . 100
2.3.11 Analysis of Inter-Burst Intervals . . . 102
2.3.12 Ripley’s Clustering Analyses . . . 102
2.3.13 Correlation Analyses for Burst Times, Minima Times, and GTEs 103 2.3.14 Analysis of Firing Rates Around Discrete Events . . . 103
2.3.15 Correlation Between Neural Activity and Acoustic Features of Song . . . 104
2.3.16 Population Analyses . . . 104
2.3.17 Covariation of Song and Neural Timing . . . 104
2.3.18 Spectral Analysis of Song Amplitude and Spike Trains . . . . 106
3 Neural Correlations in a Cortical Circuit that Generates Behavioral Variability 107 3.1 Results . . . 108
3.2 Experimental Procedures . . . 119
3.2.1 A Novel Microdrive for Population Recordings in Freely Behav-ing Birds . . . 119
3.2.2 Subjects . . . 162
3.2.3 Neural Recordings . . . 163
3.2.4 Song Recordings . . . 164
3.2.5 Song Modulation of Neural Activity . . . 164
3.2.6 Locking of Neural Activity to Song . . . 165
3.2.7 Noise Correlations . . . 165
3.2.8 Modeling Neural Responses . . . 165
3.2.9 Noise Cross-Correlations . . . 166
4 Functional Calcium Imaging in a Cortical Circuit that Generates Behavioral Variability 169 4.1 Results . . . 170
4.2 Discussion . . . 176
4.3 Experimental Procedures . . . 178
4.3.1 A Novel Endoscope for Functional Calcium Imaging in Freely Behaving Birds . . . 178
4.3.2 Subjects . . . 197
4.3.3 Functional Calcium Imaging . . . 197
4.3.5 Noise Correlations . . . 199
4.3.6 Modeling Neural Responses . . . 199
5 Discussion 201 5.1 Premotor Coding Underlying Learned Behaviors . . . 201
5.1.1 Sequential Coding in Premotor Brain Regions . . . 202
5.1.2 Flexibility of Learned Behaviors . . . 204
5.2 Premotor Coding Underlying Exploratory Behaviors . . . 205
1-1 Aspects of Movement are Encoded by Motor Cortex . . . 27
1-2 The Process of Song Imitation . . . 29
1-3 The Song System . . . 34
1-4 Sparse Coding in HVC . . . 38
2-1 A Large Dataset of HVC Neurons Recorded in Singing Birds . . . 50
2-2 Activity of HVC Projection Neurons . . . 52
2-3 Fraction of Song Motif Covered by Bursts . . . 53
2-4 Fraction of the Song Motif Covered as a Function of Fold Coverage . 54 2-5 Complete Coverage of the Song Motif as a Function of Fold Coverage 55 2-6 Quantification of Clustering in HVC Activity . . . 56
2-7 Bird-Specific Quantification of Clustering in HVC activity . . . 58
2-8 Gesture Trajectory Extrema (GTE) analysis . . . 60
2-9 Comparison of GTEs Calculated with Different Methods . . . 61
2-10 Burst Clustering Under GTE Model with Different Methods of Calcu-lating GTEs . . . 62
2-11 Projection Neuron Bursts and Interneuron Minima by Bird . . . 63
2-12 Interneuron Firing Rates Around Projection Neuron Bursts . . . 63
2-13 Variations in Density of HVC Activity Around Discrete Temporal Features of Song . . . 64
2-14 Alignment of Projection Neuron Bursts to GTEs Calculated as per Boari et al. . . 66
2-15 Locking of HVC Activity to Discrete Temporal Features of Song, by
Bird and Projection Target . . . 66 2-16 Alterative Metric of HVC Activity Alignment to GTEs, by Bird and
Projection Target . . . 68
2-17 A Simple Model of how Spike Timing Variability Drives Behavioral Variability . . . 69
2-18 Validation of Method to Analyze Relation Between HVC Spike Timing
and Song Timing . . . 70
2-19 Covariation between HVC Spike Timing and Song Timing . . . 71 2-20 Covariation of HVC Burst Activity with Song Spectral Features . . . 72
2-21 Projection Neuron Activity During Low-Freqency Harmonic Elements
(LHEs) . . . 72
2-22 Covariation of HVC Firing Rates with Song Spectral Features . . . . 73
2-23 Rhythmic Modulation of HVC Interneuron Firing Rates Locked to Syllable Structure . . . 76
2-24 Rhythmic Modulation of HVC Projection Neuron Firing Rates Locked to Syllable Structure . . . 77
2-25 Analysis of Syllable-Related Interneuron Firing Rate Modulations, by
Bird . . . 78
2-26 Analysis of Syllable-Related Projection Neuron Firing Rate Modula-tions, by Bird . . . 79
2-27 HVC Activity in Juvenile Birds . . . 81
2-28 Firing Rates of HVC Interneurons in Juvenile Birds . . . 83
2-29 Rhythmic and Non-Rhythmic HVC Interneurons in Juvenile Birds . . 83
2-30 Rhythmic Activity in Projection Neuron Subtypes in Juvenile Birds . 85
3-2 Undirected Singing Modulates Spiking Statistics in LMAN . . . 110
3-3 Single Units in LMAN Exhibit Motif-Locked Activity . . . 111
3-4 Single Units in LMAN Exhibit Syllable-Locked Activity . . . 112
3-5 Song Modulation vs. Song Locking in LMAN . . . 112
3-6 Some Pairs of LMAN Neurons Exhibit Coordinated Activity . . . 114
3-7 Correlated Activity in LMAN as a Function of Distance . . . 115
3-8 Preventing the Same Unit from Being Detected on Different Electrodes 118 3-9 Noise Time-Lag Cross-Correlation between Pairs of LMAN Neurons . 120 3-10 Noise Time-Lag Cross-Correlation vs. Noise Pearson Correlation . . . 121
3-11 Headstage Schematic Part 1 of 2 . . . 122
3-12 Headstage Schematic Part 2 of 2 . . . 123
3-13 Printed Circuit Board for Recording 64 Neural Inputs . . . 124
3-14 Chip on Board Assembly for RHD2164 . . . 126
3-15 Photo of Bonded RHD2164 . . . 127
3-16 Photo of Assembled Headstage . . . 127
3-17 Flexible Interconnect for Metal Microprobes . . . 130
3-18 Flexible Interconnect Connected to the Headstage and Metal Microprobes.131 3-19 Array of 21 Metal Microprobes . . . 131
3-20 Novel High-Density Flexible Probe . . . 133
3-21 Tip Electrodes of Flexible Probe . . . 135
3-22 Assembled Flexible Probe . . . 135
3-23 Schematic for Torque-Sensing Circuit . . . 138
3-24 Torque-Sensing PCB . . . 140
3-25 Photo of Torque-Sensing PCB . . . 142
3-26 Schematic of Torque Breakout Board . . . 144
3-27 Torque Breakout PCB . . . 146
3-28 Schematic of Electrophysiology Breakout Board Part 1 of 2 . . . 148
3-30 Electrophysiology Breakout PCB . . . 151
3-31 Schematic of Negative Feedback Controller . . . 153
3-32 Negative Feedback Controller PCB . . . 154
3-33 Motorized Variant of the Microdrive . . . 156
3-34 Photo of Customized Gearbox and Linear Actuator . . . 157
3-35 Photo of Flexible Driveshaft . . . 157
3-36 Custom Software to Remotely Position Electrodes . . . 158
3-37 Renderings of 3D Printed Case for our Microdrive . . . 160
3-38 Assembled Microdrives . . . 160
4-1 Population of LMAN Neurons Imaged During Song . . . 170
4-2 Simultaneously Imaged Activity in Bird 2 . . . 172
4-3 Song Modulation of Cells Considered to be Part of LMAN . . . 174
4-4 Correlated Calcium Activity in LMAN as a Function of Distance . . . 175
4-5 Groups of Correlated Neurons in LMAN . . . 176
4-6 Groups of Correlated LMAN Neurons in Bird 2 . . . 178
4-7 Featherscope Imaging Schematic: Overview . . . 181
4-8 Featherscope Imaging Schematic: Image Sensor . . . 182
4-9 Featherscope Imaging Schematic: Serializer . . . 183
4-10 Featherscope Imaging Schematic: Microcontroller . . . 184
4-11 Featherscope Imaging Schematic: Coax Filtering . . . 185
4-12 Featherscope Imaging Schematic: Power . . . 186
4-13 LED Driver Schematic . . . 187
4-14 The Featherscope: a 1.1 Gram Fluorescence Microscope . . . 189
4-15 A Novel Compact Optical System for Fluorescence Imaging . . . 191
4-16 Pixel Noise of Featherscope . . . 195
4-17 3D Printed Case for Featherscope . . . 196
2.1 Number of HVC Neurons Recorded in Birds with Mature Songs . . . 94
3.1 Bill of Materials for Headstage . . . 125
3.2 Bill of Materials for Torque Sensing Circuit . . . 139
3.3 Bill of Materials for Torque Breakout PCB . . . 143
3.4 Bill of Materials for Electrophysiology Breakout Board . . . 145
3.5 Bill of Materials for Negative Feedback Controller . . . 150
3.6 Number of LMAN Neuron Pairs Recorded in Each Bird . . . 162
4.1 Bill of Materials for Featherscope Imaging Board . . . 188
4.2 Number of LMAN Neuron Pairs Imaged in Each Bird . . . 197
ADC analog-to-digital converter AFF average fundamental frequency AFP anterior forebrain pathway CAN controller area network
CDF cumulative distribution function
CMOS complementary metal oxide semiconductor CPU central processing unit
DC direct current
DLM dorsal lateral nucleus of the medial thalamus DM dorsal medial nucleus
DMP descending motor pathway DPH days post hatch
EEPROM electronically erasable programmable read-only memory EI excitatory-inhibitory
ENIG electroless nickel immersion gold FDR false discovery rate
FPS frames per second GABA γ-aminobutyric acid GLM generalized linear model
GRIN gradient-index
GTE gesture trajectory extrema HPS harmonic product spectrum HVC used as a proper noun HVCi putative HVC interneuron
HVCp putative HVC projection neuron
HVCRA RA-projecting HVC neuron
HVCX Area X-projecting HVC neuron
I2C inter-integrated circuit
IBI population interburst interval IC integrated circuit
IFR instantaneous firing rate IN interneuron
ISI interspike interval KS Kolmogorov-Smirnov LED light-emitting diode
LHE low-frequency harmonic element
LMAN lateral magnocellular nucleus of the anterior nidopallium LSb least significant bit
LVDS low-voltage differential signaling MCU microcontroller unit
MSN medium spiny neuron NA numerical aperture
nXIIts tracheosyringeal part of the hypoglossal nucleus PAm nucleus para-ambiguus
PCB printed circuit board
PETH peri-event time histogram PN projection neuron
pre-SMA pre-supplementary motor area PSF point spread function
PWM pulse-width modulation
RA robust nucleus of the arcopallium RAm nucleus retroambiguus
SCI spectral content index SMA supplementary motor area SPI serial peripheral interface
USART universal asynchronous/synchronous receiver/transmitter ZIF zero insertion force
Introduction
T
hrough diligent practice, humans are able to perform an incredible range of behaviors. Whether it is a publicly celebrated feat of motor control, such asthe piano playing of Marc-André Hamelin or the tennis serve of Steffi Graf, or if it is instead a mundane activity like tying our own shoe laces or even uttering
the word hello, most of the things that people can do are learned. Through what means do we acquire these motor skills, and once they are learned how does our brain
orchestrate the execution of complex motor skills that comprise a sequence of many individual movements?
In this thesis I attempt to address these questions by studying how neural activity
in premotor brain regions generates either the exploratory behavior that is required to first learn a motor skill, or instead the execution of a skill after it is finally learned. I do
so by using song learning in the zebra finch as a lens to understand the acquisition and production of motor skills more generally. In this chapter I will provide background
information on motor learning and execution, explain the choice of using the songbird as a model organism to study these topics, identify some outstanding questions about
generating either exploratory or skillful behaviors, and finally summarize my findings on these topics.
1.1
Acquisition and Execution of Complex Motor
Behaviors
Producing motor behaviors require animals to produce a sequence of movements that evolves through time. Most animals have a repertoire of such behaviors that they
need to survive in their environment. Some of these behaviors are innate, and can be executed without first being learned. Despite not being learned, these innate
behaviors—called motor programs—can be just as complex as learned behaviors. This is the case with the waggle dance of honey bees that communicate where food sources
are located (Frisch, 1974), the courtship dance of stickleback fish (Tinbergen, 1951), self-grooming in rodents (Aldridge and Berridge, 1998), or even the production of
sounds that convey emotion such as laughter in humans (Sauter et al., 2010; Scheiner et al., 2006).
The neural basis underlying a subset of these innate behaviors, specifically the rhythmic ones, is fairly well understood. The brain regions responsible for
pattern-ing these rhythmic behaviors are called central pattern generators, and have been extensively studied in both vertebrates and invertebrates. Some prominent examples
include chewing in lobsters (Miller and Selverston, 1982), swimming in both lamprey fish (Grillner et al., 1987) and leeches (Stent et al., 1978), or the flight of locusts
(Wilson, 1961). However, both rhythmic and non-rhythmic innate motor programs are inflexible, and lack the versatility that we often associate with vertebrate behavior.
Many complex motor behaviors produced by vertebrates are not innate but are
instead learned (Thelen, 1995), including many of the behaviors that we closely associate with humans such as language or tool use (Meltzoff et al., 2009; Locke, 1993).
Despite this association, learned motor skills are not unique to humans, and motor learning has been observed in the wild in a number of species. Some species exhibit
culturally transmitted tool use in the wild: otters use stone anvils to open mussels (Mann and Patterson, 2013), while chimpanzees crack the shell of nuts with stones
(Luncz et al., 2015) and consume honey and bugs with either leaves or sticks (Gruber et al., 2009). Other species teach hunting skills to their young, such as adult meerkats
instructing juveniles on how to hunt and eat scorpions (Thornton and McAuliffe, 2006). Finally, non-human vocal imitation provides many examples of learned motor
skills in different species, and has been observed in cetaceans (Reiss and McCowan, 1993), bats (Knörnschild et al., 2006), as well as songbirds (Marler and Tamura, 1964).
While diversity of learned motor behaviors may seem to suggest that they have little in common, they share some similarities.
Learned motor behaviors often share a similar developmental trajectory, and start
with highly variable exploratory versions of the behavior, and eventually becomes stereotyped as the behavior is learned. Human speech is a familiar and prototypical
example: babies first begin by babbling highly variable vocalizations, which over time develops structured and repeated sounds. These sounds become flexibly ordered,
and finally converge on imitations of words (Lipkind et al., 2013; R. E. Stark, 1980; Oller, 1980). A similar pattern is seen with sign language, which also begins with a
highly variable babbling phase (Petitto et al., 2004). This pattern of an initial phase of variable, exploratory behavior that transitions into stereotyped production is not
specific to language, but is also present in the grasping behavior of babies (Wallace and Whishaw, 2003), as well as when babies first play with a new toy such as an
elastic harness which allows them to jump up and down (Goldfield et al., 1993). Nor is it specific to humans: dolphins (Reiss and McCowan, 1993), bats (Knörnschild et al.,
2006), and songbirds (Marler, 1970) all babble in the initial stages of vocal imitation. While most of these examples are in juvenile animals, adult parrots will start learning
new vocalizations by producing highly variable vocalizations (Pepperberg et al., 1991).
Studying the neural basis for both the acquisition and production of these learned motor behaviors is easiest to accomplish in a laboratory setting (with some impressive
exceptions, such as S. Hoffmann et al., 2019). While this has been done in non-human primates by teaching them to perform a sequence of button presses or reaches
(Hikosaka et al., 1999), and in rats doing a reaching task (Kawai et al., 2015), these behaviors are not ethologically relevant, and may engage different learning processes
than those used in naturalistic learning. In contrast, the natural motor learning of songbirds can be studied in a laboratory setting, and is a uniquely well suited to offer
insight into how complex motor behaviors are acquired in vertebrates. As studying song learning forms the basis for this thesis, their learning behavior will be introduced
in more detail in section 1.2.
1.1.1
The Neural Basis for Exploratory Behavior
What is the purpose of the behavioral variability seen at the outset of learning a
motor behavior, and what is its neural basis? One theory is that initially variable behavior reflects the immature activity of the brain region that will eventually control
the learned behavior, and that this variability is merely a poor attempt at producing the desired behavior (Edelman, 1978; Marler, 1997). However, this view is inconsistent
with causal experiments in the songbird which demonstrated that their vocal babbling is generated by a different set of brain regions than the ones that eventually produce
the learned behavior (Aronov et al., 2008; Aronov et al., 2011).
The active generation of this highly variable exploratory behavior suggests that it
plays an important role in the process of learning motor behaviors. Variability and exploration play a key role in reinforcement learning (Sutton and Barto, 1998), which
in turn has been hypothesized to be the algorithm underlying songbird vocal motor learning (Doya and T. Sejnowski, 1995; Fiete et al., 2007; Fee and Goldberg, 2011).
More specific models of how behavioral variability is generated in the brain have been proposed in songbirds, which will be discussed later. Outside of song imitation,
behavioral variability has mostly been viewed as something that the motor system seeks to minimize (Osborne et al., 2005; Churchland, Afshar, et al., 2006; Churchland,
B. M. Yu, et al., 2006), with only a few neural studies investigating how exploratory behavior is modulated to facilitate learning (Barnes et al., 2005; Sheth et al., 2011),
Figure 1-1: Aspects of Movement are Encoded by Motor Cortex.
Activity from a single neuron in the motor cortex of a rhesus monkey as the subject moved a mechanical arm in trained directions (shown in the center ), with ticks indicating action potentials relative to movement onset (M ), and each row representing a trial. The activity of this cell is strongly modulated by reach direction. Figure reproduced from Georgopoulos et al., 1982.
or behavioral studies linking variability to learning rates (Wu et al., 2014).
1.1.2
The Neural Basis for Execution of Learned Motor
Be-haviors
What is the neural basis of motor skills once they have been learned? Early research in
non-human primates executing trained wrist movements (Evarts, 1968), or directional reaches (Georgopoulos et al., 1982), suggested that motor cortex encodes either
kinematic features of movement (Figure 1-1), or instead aspects of muscle control: such as force. This kinematic view of motor cortical coding has become predominant
(Georgopoulos et al., 1986; Todorov, 2000; Paninski et al., 2004; Sergio et al., 2005; Hochberg et al., 2006; Kalaska, 2009; Griffin et al., 2015). However, these kinematic
models do not address how sequences of motor gestures are produced or appropriately timed. That aspect of motor control may be handled in higher order premotor cortical
regions (Nakamura et al., 1998; Mita et al., 2009).
Recently a number of studies have suggested that beyond just representing features of motor output, motor regions may act as their own pattern generators and contain
intrinsic oscillatory dynamics (Churchland et al., 2012), sequential dynamics (Mita et al., 2009; Murakami et al., 2014), or activity that is explicitly related to the task
sequence (Nakamura et al., 1998). These types of activity would allow motor regions to produce the correct sequence of motor gestures with appropriate timing. Confusingly,
causal manipulations of these brain regions show they are mostly required for the acquisition of these behaviors, and not execution (Nakamura et al., 1999; Kawai et al.,
2015). In contrast, the cortical nuclei in songbirds that are required for the production of learned song behavior are known. This is just one of many reasons why songbirds
represents a uniquely powerful model organism to study both the neural basis of both highly variable behavior as motor skills are initially acquired, and the production of
the behavior once it is learned.
1.2
The
Songbird
as
a
Model
Organism
for
Learned Behaviors
Adult male zebra finches (Taeniopygia guttata) sing a single song that is highly
stereo-typed, and is fittingly called crystallized song (Figure 1-2). The song is hierarchically organized, and is composed of differing sounds, or notes, that are uttered in continuous
sequences roughly 100 ms in duration, called syllables (Konishi, 1985), separated by brief periods of silence. These syllables are sung in a stereotyped sequence, called a
motif, that the bird will sing a number of times in succession, called a bout (Zann,
1996).
1kHz Motif
Tutor
Pupil
100ms 100ms Subsong Plastic Song Adult Song (Imitation)Figure 1-2: The Process of Song Imitation.
The path that juvenile zebra finches take to imitate the song of their tutor.
Upper half: The song of an adult tutor bird is shown as a sound spectrogram,
demon-strating the hierarchical nature of zebra finch song. Spectral power in each frequency band (vertical axis, frequency range 1 kHz–8 kHz) is shown as a function of time
(hori-zontal axis), with darker shading indicating more spectral power. Syllable segments are
indicated as colored bars above the sound spectrogram, with each color corresponding to a distinct syllable type. The repeated sequence of syllables is called the song motif, and is indicated with a black bracket.
Lower half: The process of song imitation as demonstrated by a single bird. Juveniles
initially sing a highly variable song akin to human babbling, called subsong, which has no repeated syllables (top spectrogram). Over the course of song learning the pupil will acquire repeated identifiable syllables, as can be seen in the plastic song stage (middle spectrogram), before finally converging on an imitation of the tutor song as the juvenile reaches sexual maturity (bottom spectrogram). This pupil was imitating the tutor song shown at the top, and the similarity of the imitation to the original can be seen by comparing the imitated song at bottom to the tutor song at the top of the figure.
adult tutor (Thorpe, 1954; Thorpe, 1958; Thorpe, 1961; Immelmann, 1969; Marler, 1970). The process of learning this song is called song imitation and occurs in two
phases: the sensory phase where the juvenile does not sing, and the sensorimotor phase where the juvenile begins to sing.
During the sensory phase, the bird listens to the tutor song and forms an auditory
memory of that song called the template memory (Eales, 1985; A. E. Jones et al., 1996). While the location and mechanism of this memory has not been directly demonstrated,
a number of observations suggest its existence. The formation of a memory of this song is clear in some songbird species that have disjoint sensory and sensorimotor
learning phases i.e. their tutor is only present before they themselves begin to sing. Even though birds in these species never hear the tutor song as they are practicing
their own song, they will still imitate the tutor song (Marler and Tamura, 1964; Marler and Peters, 1981; Marler and Peters, 1982). Similarly, zebra finches that are allowed
to hear their tutor during the sensory period, but are isolated from the tutor before they begin to sing, still learn to imitate the tutor song (Eales, 1985; Böhner, 1990;
A. E. Jones et al., 1996). If birds are instead isolated from the tutor during both learning phases, they will never develop a normal song, and will have an abnormal,
variable song well into adulthood (Price, 1979; Williams et al., 1993). Juveniles that are raised in isolation but given access to an audio recording of the tutor song will
still imitate the tutor song and develop normally (Adret, 1993; Houx and Ten cate C, 1999; Tchernichovski et al., 1999), suggesting that access to the song per se is
the necessary component for song imitation. This auditory memory is remarkably long-lasting: birds that are prevented from hearing their own song until 70–170 days
after tutoring will nonetheless learn to imitate syllables from their tutor song when they regain normal hearing (Funabiki and Konishi, 2003).
During the sensorimotor phase the bird tries to imitate the tutor song through a
process of vocal experimentation and auditory self-evaluation (Konishi, 1985). Song imitation requires juvenile birds to discover the sequence of motor commands that will
reproduce the sound of the tutor song. Juvenile birds are thought to accomplish this by comparing their own song to the template memory. Indeed, if birds are prevented
from hearing their own song during the sensorimotor phase by either deafening them after tutor exposure (Konishi, 1965; Konishi, 1964; Scott et al., 2007), or instead
playing loud masking noise over their song (Funabiki and Konishi, 2003), they will not form a good imitation the tutor song.
The sensorimotor phase has several distinct stages that span roughly 60 days in
total. It begins around 30 days after hatching when the bird begins to sing subsong—a highly variable song similar to the babbling of human babies (Figure 1-2) (Marler,
1970; Doupe and Kuhl, 1999; Lipkind et al., 2013). After a few days of subsong, most birds will start to sing a single identifiable and repeated syllable, and enter the
protosyllable phase (Tchernichovski et al., 2001; Liu et al., 2004). In a few more days,
multiple distinct syllables will emerge, marking the beginning of the plastic song stage
(Tchernichovski et al., 2001; Liu et al., 2004). Finally, 80–90 days after hatching, the song crystallizes into the highly stereotyped and precise sequence of syllables typical
of adult song motifs (Zann, 1996).
The slow and gradual nature of song imitation has led to a number of models in which song imitation occurs through a reinforcement learning process (Doya and T.
Sejnowski, 1995; Fiete et al., 2007; Fee and Goldberg, 2011). Over the course of the 60-day sensorimotor period, juveniles will sing thousands of bouts of song (Johnson et al.,
2002), slowly changing their song to make it more similar to the template (Immelmann, 1969; Tchernichovski et al., 2001). The large number of renditions and gradual changes
in song are consistent with the zebra finch learning to imitate the tutor song through a reinforcement learning process (Sutton and Barto, 1998). A neurophysiological
study (Gadagkar et al., 2016) has identified song-related performance signals in the dopaminergic midbrain of the zebra finch that are consistent with reward prediction
errors—a hallmark of reinforcement learning (Sutton and Barto, 1998)—bolstering the view that songbirds use reinforcement learning to imitate the tutor song.
Other models (Hanuschkin et al., 2013) have suggested that instead of following reinforcement learning, birds imitate the tutor song in a two-step process. First,
during the highly variable subsong phase, birds develop an inverse model which maps sounds to the motor commands required to produce that sound. Subsequently, the
inverse model is combined with the template memory to arrive at an imitation of the tutor song.
After the song is learned, songbirds will sing it in a slightly different manner depending on the social context. In the presence of a female bird, male birds will
sing directed song to the female which is used as a mating display, while in isolation they will sing undirected song which is more variable than its directed counterpart
(Sossinka and Böhner, 1980; Kao et al., 2005; Ölveczky et al., 2005; Kao and Brainard, 2006; Stepanek and Doupe, 2010; Ali et al., 2013).
The process of song learning has been compared to how human infants learn to
speak, due to the many similarities between the two processes (Marler, 1970; Doupe and Kuhl, 1999; Webb and Zhang, 2005; Lipkind et al., 2013). The parallels between
vocal learning in both songbirds and humans, despite having no common ancestor that does vocal imitation (Jarvis, 2004), suggests that these similarities may reflect
universal features of motor learning in animals, and that songbirds are therefore a good model organism to study skilled motor behaviors more generally.
What is the neural basis for this process of song imitation, and the production of the song once it is learned? Unlike the formation and storage of the template
memory remain, a fair amount is known about the neural basis of song learning and production.
1.3
The Song System: a Network of Nuclei for
Learning and Producing Song
Decades of research have identified a set of brain regions that are required for learning
the song and producing that song once it is learned. In some songbird species such as the zebra finch, singing is a sexually dimorphic behavior performed only by
males (Immelmann, 1969). Early research compared the brains of male and female birds, and hypothesized that brain regions larger in males might be involved in song
production and learning (Nottebohm and Arnold, 1976). Later lesion studies confirmed this hypothesis (Nottebohm et al., 1976; Simpson and Vicario, 1990). These brain
regions were divided into two pathways based on their connectivity, and whether their elimination affected either song learning or song production. The descending motor
pathway (DMP) is necessary for producing learned vocalizations (Nottebohm et al.,
1976; Simpson and Vicario, 1990; Aronov et al., 2008), while the anterior forebrain
pathway (AFP) is necessary for learning those vocalizations (Bottjer et al., 1984;
Scharff and Nottebohm, 1991). Collectively, these pathways are referred to as the
song system (Mooney, 2009).
The Descending Motor Pathway
The descending motor pathway controls both respiratory centers and motor neurons
that innervate the vocal organ. It comprises the forebrain nuclei HVC (used as a proper noun) and robust nucleus of the arcopallium (RA), the midbrain vocal-motor dorsal
medial nucleus (DM), medullary respiratory centers nucleus retroambiguus (RAm) and nucleus para-ambiguus (PAm), as well as the tracheosyringeal part of the hypoglossal nucleus (nXIIts) (Figure 1-3). RA is a cortical-like nucleus with homologies to layer
V of mammalian motor cortex (Karten, 1991; Jarvis, 2004; Pfenning et al., 2014),
and sends a direct glutamatergic projection to DM, RAm, PAm, as well as motor neurons in nXIIts that innervate the vocal organ, called the syrinx (Nottebohm et al.,
excitatory inhibitory
descending motor pathway
nXIIts syrinx
DLM RA
HVC
anterior forebrain pathway
LMAN Area X
pallium
(cortex)
basal
ganglia
thalamus
brainstem
DM and respiratory centersFigure 1-3: The Song System. A portion of the descending motor pathway is shown in green, and the anterior forebrain pathway is shown in orange. Both pathways contain HVC, which has distinct populations of neurons projecting either to the DMP shown in green, or the AFP shown in orange. Interneurons in HVC are shown in purple.
1982; Wild, 1993; Wild et al., 2000; Wild, 2004). RA contains a myotopic map of the musculature of the syrinx, as the projection from RA to nXIIts is topographic
(Vicario, 1991), and nXIIts is itself myotopically organized (Vicario and Nottebohm, 1988).
The DMP is necessary for the production of adult song, and is thought to produce a sequence of motor commands that produce the learned song. Lesion studies have
demonstrated that RA is necessary for both learned adult vocalizations and juvenile babbling (Nottebohm et al., 1976; Simpson and Vicario, 1990; Fukushima and Aoki,
2000; Aronov et al., 2008). In contrast, lesions and fiber transections of the downstream nXIIts and DM affect innate, learned, and juvenile vocalizations, which suggests that
RA is specialized to produce learned vocalizations. Recordings of RA neurons in singing adult birds revealed that they reliably burst at a few stereotyped times during
the song motif, with different neurons bursting at different times (A. C. Yu et al., 1996; Chi and Margoliash, 2001; Leonardo and Fee, 2005). However, using mild
cooling to change the internal dynamics of RA does not change the timing of the learned song (Long and Fee, 2008), suggesting that RA may inherit this pattern of
activity from its inputs. The majority of glutamatergic input to RA comes from two premotor forebrain nuclei: HVC and the lateral magnocellular nucleus of the anterior
nidopallium (LMAN) (Nottebohm et al., 1976; Bottjer et al., 1989; Vu et al., 1994;
Kao et al., 2005; Garst-Orozco et al., 2014). Of these two inputs, only HVC is required
for adult learned vocalizations (Aronov et al., 2008; Nottebohm et al., 1976; Simpson and Vicario, 1990), and is therefore considered to be part of the DMP. HVC will be
discussed in more detail in section 1.4, and LMAN will be discussed in section 1.5.
The Anterior Forebrain Pathway
The anterior forebrain pathway is a basal ganglia-corticothalamic loop that forms an
alternative route between the forebrain nuclei HVC and RA. The AFP comprises HVC, the basal ganglia region Area X, the dorsal lateral nucleus of the medial thalamus
(DLM), and LMAN (Figure 1-3). Like the DMP, the AFP also begins in HVC—
however the set of cells that project to Area X (which is in the AFP) are distinct
from those that project to RA (which is in the DMP) (Fortune and Margoliash, 1995; Dutar et al., 1998; Kubota and Taniguchi, 1998; Mooney, 2000; Daou et al., 2013).
Area X is the song-related portion of the basal ganglia, and contains both striatal and pallidal neurons (Farries and Perkel, 2002; Carrillo and Doupe, 2004; Person et al.,
2008; Goldberg et al., 2010; Goldberg and Fee, 2010). Long-range GABAergic pallidal neurons in Area X form strongly inhibitory synapses onto cells in DLM (Bottjer
et al., 1989; Luo and Perkel, 1999a; Luo and Perkel, 1999b), which in turn send a glutamatergic projection to LMAN (Bottjer et al., 1989; Boettiger and Doupe, 1998).
LMAN acts as the output nucleus of the AFP, and sends a glutamatergic projection to both RA and Area X (Nottebohm et al., 1982; Bottjer et al., 1984; Bottjer et al.,
1989; Mooney, 1992; Vates and Nottebohm, 1995; Luo et al., 2001; Garst-Orozco et al., 2014). The LMAN projection to RA is largely mediated by NMDA in adult birds
(Mooney, 1992; L. L. Stark and Perkel, 1999; Ölveczky et al., 2005; Charlesworth et al., 2012). Finally, the AFP receives input from dopaminergic regions in the midbrain
(Person et al., 2008), which send dense projections to Area X.
The AFP is required for learning and is thought to drive motor exploration.
Lesions in Area X or LMAN abolish song learning, but do not impact production of the learned song (Bottjer et al., 1984; Sohrabji et al., 1990), and lesions in DLM or
LMAN abolish production of juvenile subsong (Aronov et al., 2008; Goldberg and Fee, 2011). Recordings in Area X reveal that it contains distinct populations of neurons
that have similar patterns of activity as cell types seen in mammalian basal ganglia, and futhermore that the activity of all cell types within it is locked to the song behavior
(Goldberg et al., 2010; Goldberg and Fee, 2010). The variability of putative pallidal neurons in Area X is modulated by social context, with more neural variability during
the production of the variable undirected song, and less neural variability during stereotyped directed song (Woolley et al., 2014). In contrast, putative medium spiny
neurons (MSNs) have temporally sparse and song-locked activity, and the variability of this activity is not affected by social context. These results are consistent with lesions
studies which demonstrate that the AFP is necessary for changes in song variability that depend on social context (Kao and Brainard, 2006; Stepanek and Doupe, 2010).
Dopaminergic input to Area X has been shown to convery reward prediction errors (Gadagkar et al., 2016), and be necessary for song learning (L. A. Hoffmann et al.,
2016; Hisey et al., 2018). As premotor activity in LMAN is a major subject of this thesis, the role of LMAN in song learning and production will be discussed in more
detail in section 1.5.
Area X, DLM, and LMAN form a basal ganglia-corticothalamic loop with strong
parallels to those found in mammals (Luo et al., 2001; Farries and Perkel, 2002; Doupe et al., 2005; Goldberg et al., 2010; Goldberg and Fee, 2010). Similar to the
basal ganglia-corticothalamic loop of mammals (Alexander et al., 1986; Alexander and Crutcher, 1990; Shipp, 2017), the projection in each stage of the basal
ganglia-corticothalamic loop in the AFP is topographic and forms a closed loop, i.e. there is a recurrent loop from neurons in LMAN back onto themselves through Area X and
DLM (Johnson et al., 1995; Vates and Nottebohm, 1995; Iyengar et al., 1999; Luo et al., 2001). This basal ganglia-corticothalamic loop is myotopically organized: the
projection from LMAN onto RA is topographic (Iyengar et al., 1999) and RA itself is myotopically organized, as described above.
Both the anatomy of the song system and the lesion studies done within it strongly
suggest that the premotor brain regions LMAN and HVC are respectively necessary for exploratory babbling and stereotyped performance of the learned song, and most
likely accomplish this through their common projection to the motor cortical region RA. What is the nature of the neural code in HVC, and how does LMAN contribute
to behavioral variability? Before addressing these questions in more detail I will first give more background on these two important premotor nuclei.
A B
Figure 1-4: Sparse Coding in HVC.
Projection neurons in HVC produce brief 5 ms–10 ms bursts of action potentials at one or few times in the song. (A) Extracellular recording of identified RA-projecting HVC neuron (HVCRA) (bottom), with simultaneously recorded vocalization (top) of
three songs motifs. The example neuron bursts once during each motif. (B) Spike raster plot of ten HVCRA neurons and two HVC interneurons recorded in one bird
during singing. Each cell is displayed in a different color, and the spikes generated by that neuron during vocalization are shown on each row. Though not shown here, the activity of Area X-projecting HVC neurons (HVCX) is similar to HVCRA neurons,
however some neurons will burst more than once in a motif. The data for this bird is analyzed as Bird 1 in chapter 2. Figure reproduced from Hahnloser et al., 2002.
1.4
The Role of HVC in Song Production and
Learning
Causal manipulations have demonstrated the importance of HVC in both the
produc-tion and learning of birdsong. Both lesions of HVC (Nottebohm et al., 1976; Simpson and Vicario, 1990; Aronov et al., 2008; Aronov et al., 2011; Veit et al., 2011) and
transections of the fiber tract between HVC and RA (Aronov et al., 2008; Ölveczky et al., 2011) eliminate the production of learned aspects of the song, and cause birds to
revert to LMAN-driven subsong. Mild local cooling (Fee and Long, 2011; Aronov and Fee, 2011) was used bilaterally to manipulate the internal dynamics of HVC (Long
and Fee, 2008; Aronov et al., 2011; Andalman et al., 2011; Goldin et al., 2013) and demonstrate that activity within HVC is important for syllable timing. Specifically,
mild bilateral cooling of HVC produced uniform slowing of the timing of notes and syllables (Long and Fee, 2008), which was not the case when downstream RA was
similarly cooled. Finally, electrical stimulation of HVC during singing led to more severe song timing abnormalities than electrical stimulation in downstream RA (Vu
et al., 1994; Vu et al., 1998; Ashmore et al., 2005; C. Z. H. Wang et al., 2008).
Neurons in HVC have extremely temporally sparse activity during song production. In addition to local inhibitory interneurons (Dutar et al., 1998; Kubota and Taniguchi,
1998; Mooney, 2000; Wild et al., 2005; Kosche et al., 2015), HVC contains at least two classes of projection neurons (Fortune and Margoliash, 1995; Dutar et al., 1998;
Kubota and Taniguchi, 1998; Mooney, 2000)—neurons projecting to RA, part of a motor pathway that innervates muscles of the vocal organ, and neurons projecting
to Area X, part of a basal ganglia-thalamocortical loop necessary for song learning (Bottjer et al., 1984; Scharff and Nottebohm, 1991; Ali et al., 2013; Charlesworth et al.,
2012). Both types of projection neurons generate brief (5 ms–10 ms) bursts of spikes reliably at one or few times with respect to the song onset, with different neurons
active at different times in the song (Figure 1-4) (Fujimoto et al., 2011; Kozhevnikov and Fee, 2007; Long et al., 2010; Prather et al., 2008).
The relatively small numbers of neurons recorded in individual birds inspired two
incompatible views of what the population of HVC projection neurons do as the bird is singing. The population of projection neurons shown in Figure 1-4 was, prior to the
work presented in chapter 2 of this thesis, the most complete picture of activity within the HVC network of a single bird. Different assumptions about how these neurons
represent the rest of the projection neurons in HVC led to two different conclusions about population activity, and therefore two sets of models. One set of models was
based on the assumption that the neurons in Figure 1-4 are a random sample from a population of neurons that burst at all times in the song and therefore form a
sequence; accordingly, any alignment to features of the song in Figure 1-4 would be coincidental. Alternatively, these bursts and their alignment with features of the song
could be an indication of how HVC encodes the song behavior, in a manner analogous to the premotor activity seen in non-human primates during reaches, as demonstrated
in Figure 1-1. These different views on the population activity within HVC lead to different models of HVC function.
1.4.1
Models of HVC activity
Continuous-Time Models
In one view, the population of HVC projection neurons bursts continuously throughout
the song—essentially forming a clock that governs the processes of song production and learning at every moment in song (Fee and Goldberg, 2011; Glaze and Troyer,
2007; Hahnloser et al., 2002; Leonardo and Fee, 2005; Long and Fee, 2008; Long et al., 2010). For example, in one simple mechanistic model of song production, a unique
ensemble of RA-projecting HVC neurons (HVCRA) bursts at each time in the song,
driving a continuous and varying pattern of activity in the downstream vocal premotor
pathway (Fee et al., 2004; Leonardo and Fee, 2005). Continuous activity in HVCRA
neurons also figures into recent models of sequence generation in which these neurons
form a synaptically connected chain that supports the propagation of bursting activity (Glaze and Troyer, 2007; Hanuschkin et al., 2011; Jin et al., 2007; Li and Greenside,
2006; Long and Fee, 2008; Long et al., 2010). Finally, a recent model of vocal learning (Fee and Goldberg, 2011) incorporates Area X-projecting HVC neurons (HVCX) that
form a continuous representation of time which allows temporal specificity in learning (Charlesworth et al., 2011; Fee and Goldberg, 2011; Ravbar et al., 2012).
HVC projection neurons; thus we refer to them collectively as continuous-time models. While the simplest instantiations of these models generally assume an equal number
of neurons participating at each time in the sequence, such a uniform density of bursts is not required for the function of continuous-time models.
Gesture Model
An alternative hypothesis has been proposed, in which HVC activity is organized
around a few time points in the song corresponding to discrete events in the trajectory of vocal control parameters (Amador et al., 2013), called gesture trajectory extremas
(GTEs). These vocal control parameters were inferred from the song by inverting a
biophysical model of the syrinx; they roughly correspond to air pressure across the
syrinx, and tension on the vibrating membrane of the syrinx (Amador et al., 2013). Amador et al. reported that bursts of projection neurons, as well as the minima of
interneuron firing rates, were tightly synchronized with GTEs (4 ms–7 ms standard deviation). Based on their observations, Amador et al. went on to conclude that HVC
projection neurons, as a population, must be silent between GTEs. These findings are incompatible with the continuous-time models described above. In addition, because
this model posits that HVC activity occurs almost simultaneous with the song features it encodes, and there must be some delay between premotor neural activity and
behavior to allow time for the action potentials to propagate along axons, Amador et al. concluded that HVC could not be premotor to vocal output. In short, the GTE
model would force a dramatic revision of our understanding of the mechanisms of song production.
This debate between HVC having sequential activity that patterns song production,
or alternatively encoding discrete features of the song, is similar to the debate in mammalian motor coding discussed in subsection 1.1.2. To briefly summarize that
debate here: motor cortical regions have traditionally been viewed as encoding kinematic or other features of motor gesture (Evarts, 1968; Georgopoulos et al., 1982;
Georgopoulos et al., 1986; Sergio et al., 2005; Hochberg et al., 2006; Griffin et al., 2015; Kalaska, 2009; Paninski et al., 2004; Todorov, 2000), but a number of recent studies
have suggested that motor regions may additionally have intrinsic oscillatory dynamics (Churchland et al., 2012) or sequential dynamics (Mita et al., 2009; Murakami et al.,
2014; Shenoy et al., 2013) to act as their own pattern generators. By addressing these questions in the song system of the songbird, with its comparatively simple anatomy
and one-to-one link between structure and behavior, we may shed light on how learned motor skills, more generally, are encoded and produced.
1.5
The Role of LMAN in Song Production and
Learning
A number of lines of evidence suggest that LMAN is not only necessary for song learning, but that LMAN is also necessary for the production of exploratory behavior.
While lesions of LMAN in adult birds do not disrupt their ability to sing, lesions in juvenile birds prevent them from developing a normal song (Bottjer et al., 1984; Scharff
and Nottebohm, 1991). LMAN is also necessary for song plasticity in adult birds: it is necessary for the song plasticity used during an operant conditioning task (Andalman
and Fee, 2009; Warren et al., 2011; Charlesworth et al., 2012), it is necessary for the plastic degredation of song after deafening (Scott et al., 2007; K. W. Nordeen and
E. J. Nordeen, 2010), and it is necessary to compensate for peripheral injury (Williams and Mehta, 1999). In addition to preventing song plasticity, lesioning or inactivating
LMAN reduces song variability at all developmental stages (Scharff and Nottebohm, 1991; Bottjer et al., 1984; Ölveczky et al., 2005; Kao et al., 2005; Kao and Brainard,
2006; Aronov et al., 2008; Hampton et al., 2009; Andalman and Fee, 2009; Stepanek and Doupe, 2010; Ölveczky et al., 2011; Aronov et al., 2011; Warren et al., 2011;
Goldberg and Fee, 2011; Charlesworth et al., 2012; Ali et al., 2013; Woolley et al., 2014), and in the highly variable subsong stage LMAN inactivation abolishes babbling
altogether (Aronov et al., 2008).
LMAN produces biased motor exploration to minimize vocal errors. The slow
pace of song imitation makes detecting and studying changes in the song as they occur difficult. An operant conditioning paradigm, in which birds change features
of their song to avoid bursts of white noise, induces rapid changes in song over the course of a few hours (Tumer and Brainard, 2007; Andalman and Fee, 2009). This
technique has revealed that LMAN changes acoustic aspects of the song to increase song performance, and that these changes will eventually be consolidated into the
DMP and become independent of LMAN (Andalman and Fee, 2009; Warren et al., 2011; Charlesworth et al., 2012; Ali et al., 2013).
Neural activity in LMAN is highly variable, and drives song variability. Recordings of LMAN neurons in birds singing highly variable subsong and plastic song reveal
that the neurons have variable activity that is mostly not stereotyped in its alignment to song (Ölveczky et al., 2005; Aronov et al., 2008), and appears to be premotor
(Aronov et al., 2008). Electrical stimulation suggests that this activity can drive vocalizations (Kao et al., 2005), and recordings in RA show that LMAN activity drives
variability in RA neurons (Ölveczky et al., 2011). The change in song variability between undirected song and directed song requires LMAN, and is accompanied by a
change in the variability of LMAN neuron activity (Kao et al., 2005; Kao and Brainard, 2006; Kao et al., 2008). Together, these experiments and observations suggest that
LMAN, as the output nucleus of the AFP, drives variability in song. This variability can be thought of as motor exploration, as it is necessary for song learning. But does
LMAN generate this variability, or does it instead inherit variability from other parts of the AFP?
While LMAN is situated in a recurrent basal ganglia-corticothalamic loop, LMAN itself is likely the source of behavioral variability, instead of Area X. The basal ganglia
has been suggested to promote behavioral exploration in mammals (Barnes et al., 2005; Sridharan et al., 2006; Sheth et al., 2011). Similarly, a number of studies have
implicated the song-related basal ganglia Area X in producing song variability (Hessler and Doupe, 1999b; Hessler and Doupe, 1999a; Sasaki et al., 2006; Leblois et al.,
2010; Goldberg et al., 2010; Kojima et al., 2013; Woolley et al., 2014; Budzillo et al., 2017; Kojima et al., 2018); however, careful lesion studies have revealed that song
variability does not depend on Area X (Goldberg and Fee, 2011; Ali et al., 2013). Moreover, in both subsong and plastic song birds, mild cooling of LMAN to change
the dynamics of its neurons slows the highly variable features of subsong (Aronov et al., 2011), suggesting that dynamics within LMAN generate variability, instead of
passively conveying it.
The large amount of convergence between LMAN neurons and the effector muscles in the syrinx that they influence poses a challenge for LMAN generating exploratory
behavior. There are ~10,000 neurons in each LMAN (Bottjer and Sengelaub, 1989) that ultimately influcence the activity of just seven muscles in the syrinx (Brackenbury,
1980), with convergence at all stages in between: for example, the projection from LMAN to RA has ~5:1 convergence (Garst-Orozco et al., 2014). If the activity of
neurons in LMAN are uncorrelated with one another, and if converging presynaptic input onto a postsynaptic neuron produces a kind of averaging of the presynaptic
inputs, as is the case in many neuronal models (e.g. Vreeswijk and Sompolinsky, 1996), then this convergence will have the effect of reducing behavioral variability. This is
because the variance of the total post-synaptic current sould decrease as the number of inputs increase, proportional to σ2/N where σ2 is the variance of each input and N
is the number of presynaptic inputs. With the large convergence ratio of nearly 1000:1 LMAN neurons to syrinx muscles, either the anatomy of, or the activity within, the
song system must solve this apparent problem to allow neuronal variability in LMAN to drive behavioral exploration.
1.5.1
Models of LMAN Activity
Two different models of how LMAN generates behavioral variability have been proposed. Both of these models assume that neuronal variability is not reliant on true randomness,
such as the random thermally driven opening and closing of channels in the membrane of neurons (Hamill and Sakmann, 1981), but instead due to interconnected neurons
in LMAN that exhibit a particular type of nonlinear dynamics that make them very sensitive to small changes in the activity in the network. These nonlinear dynamics
that are sensitive to initial conditions are called chaotic dynamics, and produce seemingly random activity, even without a true source of randomness.
The first model of LMAN (Darshan et al., 2017) is based on a predominant
recurrent model of cortical activity, a balanced EI (excitatory-inhibitory) network (Vreeswijk and Sompolinsky, 1996). Balanced EI networks recapitulate many aspects
of cortical recordings (Vogels et al., 2005), and result in a population of neurons with asynchronous activity, with each neuron producing irregular Poisson like spiking
activity (Vreeswijk and Sompolinsky, 1996; Vreeswijk and Sompolinsky, 1998; Brunel, 2000). In these models, neurons in the same network have activity that is minimally
correlated: with the average pair-wise correlation between units is proportional to 1/N , where N is the number of units in the network (Vreeswijk and Sompolinsky,
1998), and the distribution of pair-wise correlations has a standard deviation of 1/√N
(Renart et al., 2010). While this prediction of small pair-wise correlations is consistent
with many cortical recordings in mammals (Vaadia et al., 1995; Renart et al., 2010), it is problematic for models of LMAN because of the ~1000:1 convergence between
LMAN and its effectors.
In order to overcome this problem of convergence, Darshan et al. modeled LMAN
as a balanced EI network that sent a projection to a second balanced EI network that represented RA. Critically, the projection between LMAN and RA in their model
included the experimentally observed topography between LMAN and RA (Iyengar et al., 1999); neurons in RA and LMAN were divided into “motor channels”, and all
