Cortical control and performance monitoring of interrupting and redirecting movements

HAL Id: hal-01518310

https://hal.sorbonne-universite.fr/hal-01518310

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Cortical control and performance monitoring of

interrupting and redirecting movements

Pierre Pouget, Aditya Murthy, Veit Stuphorn

To cite this version:

Pierre Pouget, Aditya Murthy, Veit Stuphorn. Cortical control and performance monitoring of in-terrupting and redirecting movements. Philosophical Transactions of the Royal Society B: Biological Sciences, Royal Society, The, 2017, 372 (1718), pp.20160201. �10.1098/rstb.2016.0201�. �hal-01518310�

Cortical control and performance monitoring of interrupting and redirecting

movements

Pierre Pouget1_{, Aditya Murthy}2 _{and Veit Stuphorn}3

1. Université Pierre &Marie Curie-Paris 6, CNRS UMR 7225, CR-ICM, UMR S975, Paris, France. 2. Centre for Neuroscience, Indian Institute of Science, Bangalore, India.

3. Department of Neuroscience and Krieger Mind/Brain Institute, Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA.

Address correspondence to:

Pierre Pouget

ICM, UMR 7225, UMRS 975.

CNRS-INSERM-Université Pierre et Marie Curie Hôpital de la Salpêtrière

47 boulevard de l'Hôpital 75651 Paris CEDEX 13 France

Voluntary behavior requires control mechanisms that ensure our ability to act independently of habitual and innate response tendencies. Electrophysiological experiments using the stop signal task in humans, monkeys, and rats have uncovered a core network of brain structures that is essential for response inhibition. This network is shared across mammals and seems to be conserved throughout their evolution. Recently, new research building on these earlier findings has started to investigate the interaction between response inhibition and other control mechanisms in the brain. Here we describe recent progress in three different areas: selectivity of movement inhibition across different motor systems, re-orientation of motor actions, and action evaluation.

The cortical networks for interrupting eye and limb movements

The specificity of control for the eye compared to other skeletal movements has been and still is a subject of debate. Theoretically, both eyes and skeletal movements have been described as relying on common code or signal for execution, but on different and/or independent processes for inhibition (Logan and Irwin, 2000). Experimentally, the neuronal activity recorded by electromyograms (EMGs) reveals similar activation prior to movement execution. However, while partial EMGs activation remains present when subjects successfully withhold limb movement responses, no such significant activation were detected when saccades are inhibited (see Figure 1 from Godlove et al. 2011). By themselves, these observations have confirmed and highlighted similarities but also potential important differences between the control of the spinal motor and the saccade systems when studying inhibition of movements.

Selectivity of movement inhibition

Whether any suppressions of movement have to exert a global inhibitory effect on other motor systems is still not totally known. Several research studies, using stop tasks for limb movements (hand, leg) as well as orofacial movements (speech), have shown that in certain contexts rapidly stopping actions can lead to global suppression of command to various motor systems (Cai et al. 2012; Khan et al. 2015; Gulberti et al. 2014). Some of these conclusions have been reached by recording cortico-spinal excitability (CSE) indexes in studies with transcranial magnetic stimulation (TMS), but also surface execution potentials (ERPs) during tasks combining the recording of one targeted and one unrelated effectors. Similar global inhibition effects were also described when cortico-spinal excitability was measure from the hand while participants successfully stop their eye movements compared to unsuccessful movements inhibition. Moreprecisely, the results have shown that around 50 ms before the estimated time at which a saccadeis successfully stopped the cortico-spinal excitability of other skeleton effectors were reducedeven if those effectors were task irrelevant. Altogether these results were interpreted as if rapidlystopping eye movements has to exert a global motor inhibitory effect (Wessel et al. 2013).

Beyond these observations, other studies rendered this feature of partial selectivity of the inhibitory process more complex than previously thought. Recent works have shown that inhibition of bilateral index finger extension or thumb abduction supports a model of inhibition of a unitary response and selective re-initiation, rather than selective inhibition (Macdonald et al. 2012). These studies indicated that successful performance in the selective condition occurred via suppression of the entire prepared response and subsequent selective re-initiation of the remaining component. Importantly, the delayed re-initiation of motor output was sensitive to the degree of similarity between responses, occurring later but at a faster rate with homogeneous

digits. Furthermore, there were persistent after effects from the selective condition on the motor system, which suggested greater levels of inhibition and a higher gain was necessary to successfully perform selective trials with homogeneous pairings (Macdonald et al. 2012). In the same vein, using a bimanual (with the two index fingers or the two middle fingers) using an inhibition task in which, one of the two finger responses should be withheld, the results indicate that a given response was slower when the response on the other hand was stopped compared to nostop condition. In addition, the given response was also made more forcefully, indicating that the requirement to stop one activated response was contaminated by the irrelevant inhibition (Ko and Miller, 2011). Also contradicting or moderating the view of a necessary broad contaminations due to inhibition activity over a large spectrum of motor effectors, some studies have shown evidence for possible selectivity of the inhibition processes (Greenhouse et al. 2012). In particular, when subjects were asked to detect suppression in the leg when the hand is being stopped, some results indicate that if subjects prepare to stop, then they were capable to do so without global effects on the motor system. Thus, with sufficient preparation time, human subjects appear to able to stop more selectively on or the other of the motor effectors (Greenhouse et al. 2012; Cai et al. 2012; Khan et al. 2015; Gulberti et al. 2014). Some but few studies go even beyond a modest selective effect by indicating that with a limited amount of training and highly compatible stimulus-response mappings, people can successfully perform a selective-stop task without any cost on the non-aborted component. Other recent studies also suggest that when participants used a global inhibition mechanism (acting alone); the selectivity is poorly active, however, participants can recruit a more selective and slower suppression mechanism when acting in social or multiple actors environment (Cavallo et al. 2014). These results demonstrate that in under certain constrains, inhibition can be selectively controlled and present a challenge for models of inhibitory control that posit the operation of generic processes (Xu et al., 2015). However and as discussed in the re-directing section, all these results also suggest that a common interrupting signal may also be at play for reach movements, at least when saccadic eye movements co-occur and strongly correlated in their reaction times. In the presence of weak eye-hand correlations, thus eye and hand control may be more specific. Altogether, these resultsquestioned the structural networks responsible to such global or selective inhibition mechanism. The previously described limitations, in our ability to use selective inhibition, have been described in association with some basic anatomical constrains attributed to the recruitment of a cortico-basal ganglia pathway that allows for the rapid inhibition of action but operates in a relatively generic manner. The refined and selective inhibition pathways have been less studied

Anatomical network supporting inhibition of movements

What are the potential sources for cortical top-down signals capable, to globally or selectively interrupt a movement? Anatomically it has been proposed that the observed global inhibition arises because rapidly stopping movements is achieved via input to the subthalamic nucleus of the basal ganglia, with a putatively broad suppressive effect on thalamocortical drive. Within these anatomical constrains an abrupt-onset stimuli is going to interrupt any ongoing processes by generating a global inhibitory motor and non-motor effects (Wessel et al. 2013). In addition, to act on a more specific and selective inhibition process, a slower fronto-striatal network activity is capable to generate activity for finely tuned control of action. Experimentally, to examine the dynamical role of each of these cortical and subcortical areas in the inhibition processes, the countermanding paradigm still provides a clear criterion for determining whether a

given neuronal activity generates signals sufficient to control the production of movements (Hanes et al. 1998). In neurophysiological practice, the key test is whether the activity of neurons is different between trials with a movement (no stop signal or non-canceled trials) and trials with no movement (canceled trials) and, critically, whether such a difference occurs before SSRT. If some neural modulation occurs after SSRT, then according to the race model that identifies SSRT with the time of inhibition of the movement, then the modulation is too late to contribute to controlling response initiation (Boucher et al., 2007; Logan and Cowan, 1984). Specifically, if a neural signal is to be sufficient to control movement then a significant difference in the activity on cancelled trials versus the activity on no stop trials, it must occur before SSRT. Within the frontal cortex, the supplementary motor area (SMA) and pre-SMA are widely considered to be of central importance for the control ability because of their role in movement initiation and inhibition. A large majority if not all movement-related neurons in SMA and pre-SMA failed to exhibit time-locked activity changes predictive of movement initiation in the context of the countermanding task. Similarly, a few inhibitory cells, responded early enough to be able to influence the cancelation of the movement. Altogether these studies have suggested that the movement-related activity in pre-SMA and SMA might be representing the motivation for a specific action but not whether or not that action is performed (Scangos and Stuphorn, 2010). Many supplementary eye field (SEF) neurons are active during the preparation and execution of saccades, in the saccade stop signal task, however, these neurons with apparent movementrelated activity fail to produce signals sufficient to control gaze (Stuphorn et al. 2010). In the same vein, previous work has reported that subthreshold microstimulation of the SEF improves stop signal task performance in monkeys by delaying saccade initiation (Stuphorn and Schall 2006). These results provide a useful perspective on a recent hypothesis that identifies the stopping process with a circuit between the presupplementary motor area, the inferior frontal gyrus, and the subthalamic nucleus (Aron et al., 2007; Mars et al. 2009). In fact, damages of the inferior frontal gyrus impaired inhibition on stop-signal trials of a countermanding task (Aron et al., 2003, 2004).

Most recent researches in rodents or primates suggest that the success in countermanding a movement is implemented via a hyperdirect pathway from frontal areas to the STN of the basal ganglia, which then activates GPi and suppresses thalamocortical drive. The broad skeletomotor suppression that occurs for outright stopping could reflect the putative divergent innervation of the GPi by the STN, for which there is some evidence from neuronal tracing studies (CITE). The engagement of this mechanism may also explain the broad skeletomotor suppression that occurs after surprising or even cognition via the same brain mechanism (Wessel et al. 2016). One possibility is that the STN-mediated impact on GPi is so divergent as to also interrupt cognitive information that is putatively maintained in the ‘associative’ cortico-basal ganglia loops. Greater activation within these cortico-basal loops, including frontal and supplementary eye fields (SEFs), and the striatum has also been observed during correctly executed redirect trials, a paradigm in which target location jumps to different places while subject is producing its movement to it (Thakkar et al. 2014). These results obtained with re-orienting paradigms will be discussed in detail in the next paragraphs.

The cortical networks for re-orienting eye and limb movements.

Although online inhibitory control is most often studied with the help of the countermanding task, a more common behavior that involves online control is one that entails

changing motor plans in dynamic environments. In this context, the double-step task has been successfully used to probe the ability to modify saccade plans (Becker and Jurgens 1979, Aslin and Shea, 1984) and reach movements (e.g. Goergopolous et al 1983; Gopal et al 2015). In this task on some random fraction of trials, a second final target appears in another location from the initial target (Fig. 8A, B). Subjects have to modify the saccade/reach plans to the initial target to make another plan to the new target. Analogous to the non-cancelled trials, in some trials subjects are unable to modify the saccade/reach plan to the initial target leading to an erroneous response. The probability of error trials, which is an index of the ability to modify the initial response, increases with the delay in the appearance of the new stimulus, and describes the compensation performance of the subject analogous to the inhibition function observed in the countermanding task.

Race model approach to study changing plans

Theoretically, the simplest model that can account for performance in a redirect task involves the use of two independent integrators—two GO-accumulators—GO1 and GO2, which represent saccade preparation to initiate an action following the onset of the initial and final target, respectively (Becker and Jürgens, 1979; Georgoplous et al 1983). However, GO-GO models fails to explain the compensation function in the redirect task (Ramakrishnan et al., 2012) in the context of switching saccade between plans. This result is because such a model does not allow for the cancellation of the saccade preparation to the first target and therefore the proportion of error trials is much more than expected. GO-GO models also predict the existence of averaging or midway saccades, that are not typically observed in the proportion expected based on the reaction time distributions of no-step trials (Camelier et al. 2009), Instead, a larger fraction of averaged saccades display prominent hypometry (Bhutani 2012; Ramakrishnan et al 2010), suggesting the presence of a STOP signal, analogous to the countermanding signal (Verbruggen et al. 2008). In addition, recent modelling work suggests the presence of a dedicated fast stop unit for fast action cancellation (Noorani and Carpenter, 2015).

Although the control of reach plans has not been studied in the race model framework as have saccades, an implicit GO-GO model continues to be the dominant framework to understand how reach plans can switch in real time (Georgopolous 1981;Soechting and Lacquaniti , 1983). The basis for this position appears to be the absence of a clear refractory period in terms of the reaction time of the second movement as well as the observation of a gradual change in the reach trajectory as well as in the EMG responses. In addition, and rather surprisingly, there have been no studies testing alternate hypotheses concerning the architecture underlying switching reach plans in the framework of the race model as has been done in the context of saccades (Camelier et al 2007; Kapoor and Murthy, 2008; Ramakrishnan et al 2012) or button presses (eg. Logan and Cowan et al. 1984). Therefore it remains unresolved at this point whether there are fundamentally different algorithms and neural mechanisms being used in eye versus hand switching?

One difference is that hand movements are slow and sensory feedback is a prominent aspect of motor control, unlike saccades that are driven predominantly by feedforward mechanisms. This notwithstanding, most of our ethologically valid reaching behaviors don’t occur in isolation but co-occur with saccadic eye movements whose reaction times are highly coordinated (Gopal et al 2015). Thus in corresponding dynamic environments it seems natural to think that the control of both should invoke similar mechanisms. This hypothesis was tested in a recent study by Gopal et al (2016) who found that performance curves were distinct for the eye

and hand when these movements were executed in isolation but were comparable when they were executed together. Second, the time to switch motor plans, called the target step reaction time (TSRT) and analogous to the SSRT, was different in the eye-alone and hand-alone conditions but was similar in the coordinated condition under the assumption of a ballistic stage of 40 ms, on average. The duration of this ballistic stage could predict the extent of eye-hand dissociations seen in individual subjects, interestingly. Finally, when subjects were explicitly instructed to control specifically a single effector (eye or hand), redirecting one effector had a strong effect on the performance of the other effector. These results suggest that a common STOP signal similar to the countermanding task may also be at play for reach movements, at least when saccadic eye movements co-occur and strongly correlated in their reaction times. In the presence of weak eye-hand correlations, however, it appears that eye and hand control may be distinct (Logan and Irwin 2000; Boucher et al 2007) suggesting the need for flexible circuits to instantiate such specific behaviours.

Neurophysiology of changing plans

Although the original motivation of the behavioral studies of the double-step saccade task was to understand the mechanisms underlying switching saccade plans (Westheimer 1954; Wheeless et al 1966; Lisberger et al 1975), interestingly, most neurophysiological investigations using the same task have been largely driven to understand different aspects ocolomotor programming such as coordinate transformations (Gnadt and Andersen 1988; Goldberg and Bruce1990; Guthrie et al.1983) or the adaptive properties of the oculomotor system (e.g., Frens and Opstal, 1997). However, such neurophysiological investigations using the double-step task within the framework of race models are critical to understand the neural mechanisms of interrupting and changing plans for three important reasons. First, what are the sources of the signals that interrupt movements emanating form? Do they reflect the same circuits that control the initiation of movements or do they reflect distinct signals from different brain regions dedicated to inhibitory control or a combination of both? Second, what are the specific representations that are subject to the interruptions? Are these representations of goals that reflect the outcome of visual selection or are these representations of movement planning or a combination of both? Given the similarity of the architectures responsible for countermanding and redirecting it is likely that the neural basis of countermanding discussed in the previous section is similar to redirection as well. Indeed, in a recent study involving search step and double-step Murthy et al (2009) showed that movement- related activity in FEF in contrast to visual neurons reflected more robustly the effect of inhibitory signals, similar to what Hanes et al (1998) described for saccade countermanding. Likewise, cells in the LIP, which is a synapse close to visual areas than FEF show little evidence of inhibitory control (Pare, personal communication), whereas it is robustly expressed in the movement- related cells in the SC (Pare and Hanes, 2003). Interestingly, a similar differential expression of inhibitory control is seen amongst cortical cells involved in the online change of reach plans. Here, premotor cortical cells have been shown to most robustly express the change in motor plans before cells in the primary motor cortex as well as the parietal cortex (Archambault et al 2011; see also Bernier et al 2011). Taken together, these evidences suggest that inhibitory signals target movement related representations to rapidly interrupt movement planning, potentially along earlier visually related signals to continuously reflect the changing goal while other sensory signals particularly in parietal cortex playing a more direct role in generating corrective movements (Desmurget et al., 1999; Archambault et al 2009 ). In the context of switching motor plans such a strategy allows for

effective stopping, yet giving adequate time for the selection of a new accurate goal representation. This strategy effectively allows for fast and brief stopping without compromising on the accuracy of the new action. Additionally the absence of a refractory period seen in single cells coding for reach/saccade plans as well as behaviorally in the reaction times of the compensated response (Nelson et al 2016, Georgopolous et al, 1983) is likely to reflect an inhibitory process that is spatially selective in nature (Ramakrishnan et al 2012).

The cortical networks for monitoring eye movements

As described in the previous sections, response interruption or re-orientation requires continuous monitoring of ongoing behaviour, its consequences, and the environmental context to detect situations when response inhibition is necessary. Neurons in the medial frontal cortex have long been known to carry signals related to response evaluation. SEF neurons respond to the anticipation and delivery of reward (Amador et al., 2000, Stuphorn et al., 2000), as well as to errors and response conflict (Stuphorn et al., 2000, Nakamura et al., 2005). Similar signals reflecting positive and negative evaluations of actions have been found in the ACC (Ito et al., 2003, Matsumoto et al., 2007). These same cortical areas also contain neurons that participate in the control and selection of motor behaviour (Stuphorn and Schall, 2006, Isoda and Hikosaka, 2007). This indicates that medial frontal cortex might be an important node in the neural circuit underlying monitoring and control of behaviour. However, many aspects of this circuit and its role in behaviour are not yet well understood. In particular, it not clear what the origin of the monitoring signals is. Furthermore, it is debated whether the role of medial frontal cortex is exclusively evaluative in nature or whether they are part of the control machinery.

Origin of monitoring signals

The monitoring signals represent mismatch between expected and actual outcomes of actions. This evaluation process is often described in terms of reinforcement learning models (Rescorla and Wagner, 1972, Pearce and Hall, 1980). A key element in many of these models is the ‘reward prediction error’, i.e. the difference between the expected and the actual reward outcome (Sutton and Barto, 1998). An important open question is the source of the signals that need to be compared and where this comparison takes place. Outcome-related neuronal activities such as the ones found in SEF and ACC are also found in other brain structures, such as the dopaminergic midbrain neurons (Matsumoto and Hikosaka, 2009) and in the amygdala (Belova et al., 2007). Both of these subcortical areas have widespread projections into the medial frontal cortex (Ghashghaei et al., 2007; Holroyd and Coles, 2002). It is therefore possible that the monitoring signals in SEF and ACC simply reflect input coming from these areas (Holroyd and Coles, 2002). An alternative possibility is that the medial frontal cortex itself computes the monitoring signals. To compute a reward prediction error signal, it is necessary to represent both its precursor signals, i.e., the expected and the actual value of the outcome. If this computation takes place in medial frontal cortex, one would expect that both the precursor and outcome signals should be present in this cortical area.

A recent study has examined this question in SEF using an oculomotor gambling task, in which monkeys choose between options with uncertain reward outcome. SEF neurons carry various monitoring signals throughout the delay, when the outcome of the choice is still unknown, and the result period (So and Stuphorn, 2012). In particular, SEF neurons represent the

expected value of the chosen option throughout the delay and the result period. Following the result, SEF neurons represent the actual reward that was received and a reward prediction error signal, i.e., the comparison of the expected and actual reward signals. Such a reward prediction error signal is equivalent to the teaching signal that is predicted in the Rescorla-Wagner model of reinforcement learning (Rescorla and Wagner, 1972), and is similar to the well-known signal carried by midbrain dopamine and habenular neurons (Schultz et al., 1997, Matsumoto and Hikosaka, 2007). Thus, these findings suggest that SEF could compute a reward prediction error signal using locally represented signals about expected and actual reward without input from other structures. Similar local computation of reward prediction error likely occurs in other cortical areas (Seo and Lee, 2007, Sul et al., 2010). All of these local computations are likely to be context and effector-dependent. For example, SEF would be expected to compute reward prediction error signals only in the context of eye movements. The outcome of these local computations could be sent the dopaminergic midbrain nuclei and the habenula via connections through the basal ganglia (Calzavara et al., 2007, Hong and Hikosaka, 2008). These converging inputs from multiple specialized evaluation systems might generate more general reward prediction error representation.

In addition to the reward prediction error-related signals, the SEF contains a number of other evaluative signals (Seo and Lee, 2009, So and Stuphorn, 2012, 2015). A particular type of monitoring signal that has received a lot of experimental and theoretical consideration is confidence. Choices are made with varying degrees of confidence, a cognitive signal representing the subjective belief in the optimality of the choice. Confidence has been mostly studied in the context of perceptual judgments, in which choice accuracy can be measured using objective criteria (Kiani and Shadlen, 2009). The oculomotor gambling task allows to study confidence in subjective value-based decisions, which have to be based on subjective criteria. The SEF contains neural signals that explicitly represent choice confidence independent from reward expectation (So and Stuphorn, 2015). This confidence signal appeared after the choice and diminished before the choice outcome (Figure 4). Most of this neuronal activity was negatively correlated with confidence, and was strongest in trials on which the monkey spontaneously withdrew his choice. This indicates that SEF not only guides saccade selection, but also evaluates the likelihood that the choice was optimal. This internal evaluation influences decisions concerning the willingness to bear later costs that follow from the choice or to avoid them.

Role of medial frontal cortex in response evaluation and response control

Lack of confidence in the choice is conceptually similar to conflict, another type of monitoring signal that has been suggested to be represented by the medial frontal cortex, specifically in the ACC (Carter et al., 1998, Carter et al., 1999, Cohen et al., 2000, MacDonald et al., 2000, Kerns et al., 2004, Yeung et al., 2004). Both low choice confidence and conflict are driven by equally strong evidence in favour of choosing among mutually exclusive actions. The main difference is that confidence evaluates the quality of the choice and therefore can only be computed following the action, while conflict monitors the ongoing degree of co-activation of mutually exclusive action preparation processes. Thus, conflict could in principle detect early signs of potential problems in ongoing action selection and be used to recruit executive control processes to resolve the problems and to avoid errors. The conflict hypothesis has been very influential, but almost all the supportive evidence is based on human neuroimaging experiments (Carter et al., 1998, MacDonald et al., 2000, Kerns et al., 2004). In practically all of the

experimental paradigms that are used in these experiments, the actual error rate in ‘high conflict’ conditions is typically low (8-5%). Instead, what is typically seen is a lengthening of reaction times which is interpreted as caused by the conflict and the need to suppress it. This interpretation is not unreasonable, but it confounds the interpretation of neuronal activity modulations that are observed during high conflict trials. This activity could represent conflict monitoring, increased control efforts, or both. Unfortunately, this problem also affects recent studies that claimed to have found explicit conflict signals represented by single neurons recorded in human ACC (Sheth et al., 2012).

An additional problem for the conflict hypothesis is the fact that single unit recording studies in monkeys have shown no evidence for conflict representation in ACC (Ito et al., 2003, Nakamura et al., 2005). One possibility for this discrepancy could be a species difference (Cole et al., 2009, Schall and Emeric, 2010). While possible, it would stand in stark contrast to a large amount of similarity that has been found in other in cognitive domains, even including metacognitive signals such as confidence. Alternatively, it might be that conflict monitoring is not a function of ACC, but of other parts of the medial frontal cortex. Potential conflict signals have been found in SEF (Stuphorn et al., 2000), but not ACC (Ito et al., 2003), using an identical stop signal paradigm. Recent human recording studies have shown that monitoring signals appear earlier in SMA than in ACC (Bonini et al., 2014). Lastly, it might be that the conflict hypothesis, at least in its original conception, is not the best conceptual framework for describing the function of medial frontal cortex. Alternative explanations for the neuroimaging findings have been suggested (Brown and Braver, 2005, Alexander and Brown, 2011). Altogether, the debate about the functional merit of the conflict hypothesis is ongoing and more experiments are necessary to clarify the issue.

Conclusions

Recent experiments has started to connect our understanding of response inhibition with many other aspects of behavioural control. This research will be important to understand how response inhibition is used and controlled itself to achieve the overall goals of an agent in its day-to-day behavior. However, many questions are still unclear. Making progress will require further investigations using the stop signal paradigm. New rodent models will allow to investigate and manipulate neural circuits in unprecedented detail. Nevertheless, experiments in behaving monkeys will likely stay at the core of this enterprise. Monkeys have exceptional behavioral flexibility, which makes them ideal models to study complex control processes. They are also the closest model of human behavior and physiology that is available. Together, these different animal models will add their different strengths and offer a bright future for this exciting field of neuroscience.

References:

Alexander WH, Brown JW (2011) Medial prefrontal cortex as an action-outcome predictor. Nat

Neurosci 14:1338-1344.

Amador N, Schlag-Rey M, Schlag J (2000) Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84:2166-2170.

Archambault PS, Caminiti R, Battaglia-MayerA. (2009) Cortical mechanisms for online control of hand movement trajectory: the role of the posterior parietal cortex. Cereb Cortex 19:2848 –2864.

Archambault PS, Ferrari-Toniolo S, Battaglia-Mayer A. (2011) Online control of hand trajectory and evolution of motor intention in the parietofrontal system. J Neurosci; 31 (2):742-52. Aron AR, Behrens TE, Smith S, Frank MJ, Poldrack RA. (2007) Triangulating a cognitive

control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. J Neurosci Apr 4;27(14):3743-52.

Aron AR, Monsell S, Sahakian BJ, Robbins TW.A (2004) Componential analysis of task-switching deficits associated with lesions of left and right frontal cortex.

Brain.Jul;127(Pt 7):1561-73.

Aron AR, Schlaghecken F, Fletcher PC, Bullmore ET, Eimer M, Barker R, Sahakian BJ, Robbins TW. (2003) Inhibition of subliminally primed responses is mediated by the caudate and thalamus: evidence from functional MRI and Huntington's disease.

BrainMar;126(Pt 3):713-23.

Aslin RN, Shea SL. (1987) The amplitude and angle of saccades to double-step target displacements. Vision Res 27: 1925–1942.

Belova MA, Paton JJ, Morrison SE, Salzman CD (2007) Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55:970-984.

Becker W and Jurgens R. (1979)An analysis of the saccadic system by means of doublestep stimuli.Vision Res 19: 967–983.

Bhutani NS, Ray A, and Murthy A. (2012) Is saccade averaging determined by visual processing or movement planning? J. Neurophysiol 108: 3161-3171.

Bonini F, Burle B, Liegeois-Chauvel C, Regis J, Chauvel P, Vidal F (2014) Action monitoring and medial frontal cortex: leading role of supplementary motor area. Science 343:888-891.

Boucher L, Palmeri TJ, Logan GD, Schall JD. (2007) Inhibitory control in mind and brain: an interactive race model of countermanding saccades. PsycholRev 114: 376 –397.

Boucher L, Stuphorn V, Logan GD, Schall JD, Palmeri TJ. (2007) Stopping eye and hand movements: are the processes independent? Percept Psychophys. Jul;69(5):785-801. Brown JW, Braver TS (2005) Learned predictions of error likelihood in the anterior cingulate

cortex. Science 307:1118-1121.

Cai W, Oldenkamp CL, Aron AR. (2012) Stopping speech suppresses the task-irrelevant hand.

Brain Lang Mar;120(3):412-5.

Calzavara R, Mailly P, Haber SN (2007) Relationship between the corticostriatal terminals from areas 9 and 46, and those from area 8A, dorsal and rostral premotor cortex and area 24c: an anatomical substrate for cognition to action. Eur J Neurosci 26:2005-2024.

Camalier CR, Gotler A, Murthy A, Thompson KG, Logan GD, Palmeri TJ, Schall JD. (2007) Dynamics of saccade target selection: race model analysis of double step and search step saccade production in human and macaque. Vision Res 47: 2187–2211.

Carter CS, Botvinick MM, Cohen JD (1999) The contribution of the anterior cingulate cortex to executive processes in cognition. Rev Neurosci 10:49-57.

Carter CS, Braver TS, Barch DM, Botvinick MM, Noll D, Cohen JD (1998) Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280:747-749. Cavallo A, Catmur C, Sowden S, Ianì F, Becchio C. (2014) Stopping movements: when others

slow us down. Eur J Neurosci. Sep;40(5):2842-9.

Cohen JD, Botvinick M, Carter CS (2000) Anterior cingulate and prefrontal cortex: who's in control? Nat Neurosci 3:421-423.

Cole MW, Yeung N, Freiwald WA, Botvinick M (2009) Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci 32:566-574.

Desmurget M, Epstein CM, Turner RS, Prablanc C, Alexander GE, Grafton ST. (1999) Role of the posteriorparietal cortex in updating reaching movements to a visual target. Nat Neurosci 2:563–567.

Frens MA, Van Opstal AJ. (1997) Monkey superior colliculus activity during short-term saccadic adaptation. Brain Res Bull 43(5):473-83.

Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT. (1983) Interruption of motor cortical discharge subservingaimed arm movements. Exp BrainRes 49:327–340.

Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT. Exp BrainRes 49:327–340, 1981.

Ghashghaei HT, Hilgetag CC, Barbas H (2007) Sequence of information processing for emotions based on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage 34:905-923.

Gnadt JW, Andersen RA. (1988) Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70: 216 –220.

Godlove DC, Garr AK, Woodman GF, Schall JD. (2011) Measurement of the extraocular spike potential during saccade countermanding. J Neurophysiol Jul;106(1):104-14.

Goldberg ME, Bruce CJ. (1990) Primate frontal eye fieldsIII. Maintenance of a spatially accurate saccade signal. JNeurophysiol 64: 489 –508.

Gopal AP andMurthy A.(2016) A common control signal and a ballistic stage can explain the control of coordinated eye-hand movements. J. Neurophysiol 115(5): 2470-84.

Gopal AP and Murthy A. (2015) Eye-hand coordination during a double-step task: evidence for a common stochastic accumulator. J. Neurophysiol 114(3):1438-54.

Gopal AP, Vishwanathan P, Murthy A. (2015) A common stochastic accumulator with effector-dependent noise can explain eye–hand coordination. J. Neurophysiol 113(7):2033-48.

Guthrie BL, Porter JD, Sparks DL. (1983) Corollary discharge provides accurate eye positioninformation to theoculomotor system. Science 221: 1193–1195.

Gulberti A, Arndt PA, Colonius H. (2014) Stopping eyes and hands: evidence for non-independence of stop and go processes and for a separation of central and peripheral inhibition. Front Hum Neurosci Feb 18;8:61.

Greenhouse I, Oldenkamp CL, Aron AR. (2012) Stopping a response has global or nonglobal effects on the motor system depending on preparation. J Neurophysiol Jan;107(1):384-92.

Hanes DP, Patterson WF, Schall JD. (1998) Role of frontal eye fields in countermanding saccades: visual, movement, and fixation activity. J Neurophysiol 79: 817– 834.

Holroyd CB, Coles MG (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109:679-709.

habenula. Neuron 60:720-729.

Isoda M, Hikosaka O (2007) Switching from automatic to controlled action by monkey medial frontal cortex. Nat neuroscience 10:240-248.

Ito S, Stuphorn V, Brown JW, Schall JD (2003) Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302:120-122.

Kapoor V and Murthy A. (2008) Covert inhibition potentiates online control in a double-step task. Journal of Vision 8:1-16.

Khan AZ, Blohm G, Pisella L, Munoz DP. (2015) Saccade execution suppresses discrimination at distractor locations rather than enhancing the saccade goal location. Eur J

NeurosciJun;41(12):1624-34.

Kerns JG, Cohen JD, MacDonald AW, 3rd, Cho RY, Stenger VA, Carter CS (2004) Anterior cingulate conflict monitoring and adjustments in control. Science 303:1023-1026.

Kiani R, Shadlen MN (2009) Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324:759-764.

Ko YT, Miller J. (2011) Nonselective motor-level changes associated with selective response inhibition: evidence from response force measurements. Psychon Bull RevAug;18(4):813-9.

Lisberger SG, Fuchs AF, King WM, Evinger LC. (1975) Effect of mean reaction time on saccadic responses to two-step stimuli with horizontal and vertical components. Vision Res 15: 1021–1025.

Logan G, Cowan WB. (1984) On the ability to inhibit thought and action: a theory of an act of control. Psycholo Rev 91: 295–327.

Logan GD, Irwin DE. (2000) Don’t look! Don’t touch! Inhibitory control of eye and hand movements. Psychon Bull Rev 7: 107–112.

MacDonald AW, 3rd, Cohen JD, Stenger VA, Carter CS (2000) Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288:1835-1838.

Macdonald HJ, Stinear CM, Byblow WD. (2012) Uncoupling response inhibition. J

Neurophysiol Sep;108(5):1492-500.

Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447:1111-1115.

Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459:837-841.

Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat neuroscience 10:647-656.

Mars RB, Klein MC, Neubert FX, Olivier E, Buch ER, Boorman ED, Rushworth MF. (2009) Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict. J Neurosci May 27;29(21):6926-31.

Murthy, A., Shorter-Jacobi, S.M., Thompson, K.G. and J. D. Schall. (2009) Neural control of visual search by frontal eye field: Effects of target displacement on visual selection and saccade preparation. JNeurophysiol 101: 2485-2507.

Nakamura K, Roesch MR, Olson CR (2005) Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J Neurophysiol 93:884-908.

Nelson M.J., Murthy A. and Schall JDS. (2016) Neural control of visual search by frontal eye field: Identification of stages of processing and neural accumulation with a race to threshold. J. Neurophysiol 115(4):1954-69.

Noorani I, Carpenter RH. (2015) Ultrafast initiation of a neural race by impending errors. J

Physiol. 593(19): 4471-84.

Paré M, Hanes DP. (2003) Controlled movement processing: superior colliculus activity associated with countermanded saccades. J Neurosci 23: 6480-6489.

Pastor-Bernier A, Tremblay E, Cisek P. (2011) Dorsal premotor cortex is involved in switching motor plans. Front Neuroeng. 4;5:5.

Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not unconditioned stimuli. Psychol Rev 87:532-552.

Ramakrishnan A., Sureshbabu Ramakrishnan andMurthy A. (2012) Understanding how the brain changes its mind: microstimulation in the macaque frontal eye field reveals how saccade plans are changed. J of Neurosci. 32(13):4457– 4472.

Ramakrishnan A, ChokandreS, and Murthy A. (2010) Voluntary control of multi-saccade gaze shifts during movement preparation and execution. J. Neurophysiol103:2400-16. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the

effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II: Current Research and Theory(Black, A. H. and Prokasy, W. F., eds), pp 64-99 New York: Appleton-Century-Crofts.

Schall JD, Emeric EE (2010) Conflict in cingulate cortex function between humans and macaque monkeys: More apparent than real. Commentary on Cole MW, Yeung N, Freiwald WA, Botvinick M (2009): Cingulate cortex: Diverging data from humans and monkeys. Trends Neurosci 32:566-574. Brain, behavior and evolution 75:237-238.

Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.

Scangos KW, Stuphorn V. (2010) Medial frontal cortex motivates but does not control movement initiation in the countermanding task. J Neurosci Feb 3;30(5):1968-82. Stuphorn V, Brown JW, Schall JD. (2010) Role of supplementary eye field in saccade initiation:

executive, not direct, control. J Neurophysiol Feb;103(2):801-16.

Stuphorn V, Schall JD. (2006) Executive control of countermanding saccades by the supplementary eye field. Nat Neurosci Jul;9(7):925-31.

Seo H, Lee D (2007) Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. The Journal of neuroscience : the official journal of the Society for Neuroscience 27:8366-8377.

Seo H, Lee D (2009) Behavioral and neural changes after gains and losses of conditioned reinforcers. The Journal of neuroscience : the official journal of the Society for Neuroscience 29:3627-3641.

Sheth SA, Mian MK, Patel SR, Asaad WF, Williams ZM, Dougherty DD, Bush G, Eskandar EN (2012) Human dorsal anterior cingulate cortex neurons mediate ongoing behavioural adaptation. Nature 488:218-221.

So N, Stuphorn V (2012) Supplementary eye field encodes reward prediction error. J Neurosci 32:2950-2963.

So N, Stuphorn V (2016) Supplementary Eye Field Encodes Confidence in Decisions Under Risk. Cereb Cortex.Feb;26(2):764-82.

Soechting JF, Lacquaniti F.( 1 9 8 3 ) Modification of trajectory of a pointing movement in response to a change in target location. J Neurophysiol49:548 –564.

Stuphorn V, Schall JD (2006) Executive control of countermanding saccades by the supplementary eye field. Nat Neurosci 9:925-931.

Stuphorn V, Taylor TL, Schall JD (2000) Performance monitoring by the supplementary eye field. Nature 408:857-860.

Sul JH, Kim H, Huh N, Lee D, Jung MW (2010) Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66:449-460.

Thakkar KN, van den Heiligenberg FM, Kahn RS, Neggers SF. (2014) Frontal-subcortical circuits involved in reactive control and monitoring of gaze. J Neurosci Jun 25;34(26):8918-29.

Verbruggen F, Schneider DW, Logan GD. (2008) How to stop and change a response: the role of goal activation in multi-tasking. J ExpPsychol Hum Percept Perform 34: 1212–1228. Westheimer G. (1954)

Eye movements to horizontally moving visual stimulus. ArchOphthalmol 52: 932–941. Wessel JR, Reynoso HS, Aron AR. (2013) Saccade suppression exerts global effects on the

motor system. J Neurophysio. Aug;110(4):883-90.

Wheeless LL, Boynton RM, Cohen GH. (1966) Eye-movement responses to step and pulse-step stimuli. J Opt SocAm 56: 956 –960.

Xu J, Westrick Z, Ivry RB. (2015) Selective inhibition of a multicomponent response can be achieved without cost. J Neurophysiol Jan 15;113(2):455-65.

Yeung N, Botvinick MM, Cohen JD (2004) The neural basis of error detection: conflict monitoring and the error-related negativity. Psychol Rev 111:931-959.

CAPTIONS FOR FIGURES

Figure 1: No saccadic spike potentials (SPs) are evident in canceled trials aligned on a virtual

saccade event. The most prominent components in the ERPs are the sharp negative SPs, which occur just before or concomitant with the saccade onset and the several positive and negative deflections that follow. Note the extreme similarity of the ERVs for no-stop and noncanceled trials. Also note the similarity between no-stop and noncanceled ERPs. This similarity is especially apparent in the time before the saccade onset when the SP is visible. (From Godlove et al. 2011)

Figure 2: Diffusion-weighted tractography results. A, 3-D rendering of the tracts between the

right IFC, the right preSMA, and the right STN region.B, Triangulation method for determining the third point in a network from the other two. Tracts originating in one brain area are overlaid on tracts originating from another. The overlap is superimposed on a gray matter mask in standard space. Tracts clearly overlap in the white matter space, but the overlap in gray matter is fairly unique: the preSMA only for tracts originating in the IFC and STN regions; the IFC and anterior prefrontal cortex (not shown) for tracts originating in the preSMA and the STN region and the thalamus only for tracts originating in the preSMA and the IFC. (From Aron et al. 2007)

Figure 3: Movement (A) and Visual activity (B) during target-step trials in the double-step task

in which the target stepped out of the receptive field. Compensated target-step trials (red solid) and latency-matched no-step trials (black). While movement related activity showed countermanding (cancellation) of partially prepared motor plan before the TSRT visual cells did not. C and D. Population response of movement and visual in the search-step and double-step tasks in which the target stepped out of the receptive field. Step response ratios, i.e., ratios of activity of (C) movement and (D) visual neurons at ±20 ms TSRT during compensated target-step trials compared with latency-matched no-target-step trials (open) and during non-compensated target-step trials compared with latency-matched no-step trials (solid). While movement activity distinguished between compensated and non-compensated responses visual activity did not.(From Murthy et al. 2009).

Figure 4: SEF neurons representing decision confidence following the choice. Economic decisions are driven by a comparison of the value of the available options. Small differences in value between the chosen and the unchosen option result therefore in low choice confidence, while large differences lead to high choice confidence. The activity of many SEF neurons is negatively (A) or positively (B) correlated with value difference, reflecting choice confidence. This confidence-related signal is present independent of the amount of reward expectation (compare activity on high and low value trials). The best regression model for each example neuron is shown to the right of the histograms. Neuronal activities (dots) are plotted against the value difference between the chosen and the unchosen option. (A) In some cases, the confidence-related signal is multiplexed with other outcome-confidence-related signals. The neuron that shows a negative correlation with value difference (representing low confidence) also shows a positive correlation with the value of the unchosen option (UV). The three lines indicate the best regression model for low (red), medium (green), and high (blue) unchosen value. (B) The neuron that shows a positive correlation with value difference (representing high confidence) shows no correlation with any other outcome-related variable. (From So and Stuphorn, 2015).

0 10 20 30 40 10 20 30 40

low value trials

time from saccade onset (ms)

high value trials

0 100 200 300 400 0 5 10 15 0 100 200 300 400 0 5 10 15

A

B

mean firing rate (Hz)

low CV med CV high CV 0 10 20 30 40 50 low UV med UV high UV firing rate (Hz)