Understanding subprocesses of working memory through the lens of model-based cognitive
neuroscience
Anne C Trutti
1,2, Sam Verschooren
3, Birte U Forstmann
1and Russell J Boag
1Workingmemory(WM)referstoasetofprocessesthat makestask-relevantinformationaccessibletohigher-level cognitiveprocesses.RecentworksuggestsWMis
supportedbyavarietyofinformationgating,updating,and removalprocesses,whichensureonlytask-relevant informationoccupiesWM.Currentneurocomputational theorysuggestsWMgatingisaccomplishedvia‘go/no-go’
signallinginbasalganglia-thalamus-prefrontalcortex pathways,butislessclearaboutothersubprocessesand brain structuresknownto playaroleinWM.Wereview recenteffortsto identifytheneuralbasisofWM
subprocessesusingtherecentlydevelopedreference-back taskasabenchmarkmeasureofWMsubprocesses.Targets forfutureresearchusingthemethodsofmodel-based cognitiveneuroscienceandnovelextensionstothe reference-backtaskaresuggested.
Addresses
1DepartmentofPsychology,UniversityofAmsterdam,Amsterdam,The Netherlands
2InstituteofPsychology,LeidenUniversity,Leiden,TheNetherlands
3DepartmentofExperimentalClinicalandHealthPsychology,Ghent University,Ghent,Belgium
Correspondingauthor:
Trutti,AnneC(r.j.boag@uva.nl),Boag,RussellJ(r.j.boag@uva.nl)
CurrentOpinioninBehavioralSciences2020,38:57–65
ThisreviewcomesfromathemedissueonComputationalcognitive neuroscience
EditedbyGeoffSchoenbaumandAngelaJLangdon
https://doi.org/10.1016/j.cobeha.2020.10.002
2352-1546/ã2020TheAuthor(s).PublishedbyElsevierLtd.Thisisan openaccessarticleundertheCCBYlicense(http://creativecommons.
org/licenses/by/4.0/).
Workingmemory and itssubprocesses
Working memory (WM) refers to a set of processes that makes task-relevant information accessible to higher-level cognitive processes such as learning, decision making, reasoning, and reading comprehen- sion [1–3]. Working memory is extremely capacity- limited, withcurrent researchsuggesting that between
oneandfouritems4canbemaintainedinanactivated stateinWMatatime[4–7].Thisstrictlimitdemandsa highdegreeofcontroloverWMcontent,suchthatWM must strikea balancebetween stability(i.e. protecting thecurrentcontentsofWMfromirrelevantordistract- inginformation)andflexibility(i.e.keepingWMup-to- date with new relevant information and removing outdatedinformation).Thistrade-offbetweenstability and flexibility [8–11] is a core feature of executive controlprocesses(e.g.cognitivecontrol,conflictmoni- toring/resolution, task switching; [12]) and managing thetrade-offstronglydependsonthebrain’sdopamine systems [13,14].
Prominent computational theories suggest that WM resolves thestability-flexibilitytrade-offbyoperating in two modes: An updating (gate-open) mode, which allowsnewinformationtoenterWM,andamaintenance (gate-closed) mode, which prevents irrelevant and distracting information frominterfering with thecur- rentcontentsofWM[15–22].Inthegate-openmode, updating is further supported by two main subpro- cesses: Item removal and item substitution, which togetherensurethatonlyrelevantinformationiskept active in WM [23,24]. Together, these processes allow WM to alternate modes between flexible(when new information is encountered) and stable (when distractors are encountered). This enables successful performance in dynamic environments in which dis- tractionsarecommonandtherelevanceofinformation frequently changes.
To-date, themostdetailedneurocomputationalaccount of the gating mechanism controlling the trade-off between updating and maintenance is the prefrontal
4Weusetheterm‘item’torefertoanindividualrepresentationheld inWM.‘Item’ isthussynonymouswith‘chunk’ [97]and‘cognitive object’[98,7]whichdenotethesameconcept.Thereisongoingdebate aboutwhetheritemsinWMarerepresentedindiscreteslots(itemsheld withhighprecisioninanumberofdiscretememorylocations),allocation of continuous resources (items allocated limited resources in inverse proportiontothetotalnumberofitemsinWM),orsomehybridof thetwoframeworks(e.g.Refs.[78,74,79,80]).Themodelsandgeneral approach thatwediscussin thispaperare notcommittedto either architecturebutcouldbeusedtotestbetweenthecompetingaccounts (seeSection‘Currentdirections’below).
cortex-basal ganglia WM (PBWM) model (Figure 1;
[25–27]).Inthismodel,gatingisimplementedviabasal ganglia (BG)-thalamus-prefrontal cortex (PFC) circuits that control ‘go/no-go’ signalling. As illustrated in Figure 1, gate opening is controlled by a striatal ‘go’
signal which inhibits substantia nigra pars reticulata (SNr) and disinhibits thalamus, which in turn excites PFC.ThisallowsinformationtoenterWMandupdating tooccur.Gateclosing5iscontrolled byastriatal ‘no-go’
signal which inhibits external globus pallidus (GPe), disinhibits SNr, and inhibits thalamus. This in turn inhibits PFC,which preventsWM from beingupdated (Figure 1;[26]).In short,the‘go’ signalpassesthrough two inhibitory connections (striatum-SNr-thalamus), which excites PFC, while the ‘no-go’ signal passes through three inhibitory connections (striatum-GPe- SNr-thalamus),whichinhibitsPFC.Thesecircuitshave alsobeenimplicatedinupdatingvaluerepresentationsin reinforcementlearningandvalue-baseddecisionmaking, suggestingageneralneuralmechanismforaccomplishing informationgating([16,28,29,26,20,30,22]).
The components of the PBWM model have received broadsupportfromfunctionalmagneticresonanceimag- ing (fMRI) studies ([19,28,31–33,22,34]). For example, activation in striatum and dorsolateral PFC has been widelyreportedintasksbroadlyinvolvingWMupdating (e.g.Refs.[31–34]),whileotherworkhaslocalizedactiv- ity specifically related to the updating and gating pro- cessesrather than other WMprocesses. Roth et al. [22]
identifiedafrontoparietalnetworkspecificallyinvolvedin updating,whileMurtyetal.[19]foundselectiveengage- mentofSN/ventraltegmentalarea(VTA),caudate,dor- solateralPFC,andsomeareasofparietalcortexrelatedto theupdatingbutnotmaintenancemodeofWM.Striatal dopamine-receptor expressing neurons and dopamine- producingmidbrainstructureshavealsobeenimplicated inWM updating [19,28,33],anddynamic causalmodel- ling suggests that BG plays a central role in gating informationtoPFC[35].Moreover,anumberofcortical areas(e.g.dorsolateralPFC,medialPFC,posteriorpari- etalcortex)havebeenlinkedtothemaintenancemodeof WM but not updating ([22,36–38]). This is consistent withtheideathattonicdopamineactivityinPFCcontrols thestabilityofWMrepresentationswhereasphasicdopa- minereleaseinthestriatumtrainstheBGwhentoopen thegate(viadisinhibitionofthalamusandPFC)toallow informationintoWM6[27].
Overall,thesefindings showthatWMupdatingengages cortico-striatalcircuitryinvolvingBG,midbrain,andPFC structures broadly in line with the neurocomputational mechanisms of the PBWM model [39,26] and more general accounts of cognitive control (e.g. Ref. [40]).
However, as will be discussed, recent work highlights thatWMalsodependsonseveralimportantsubprocesses notaccountedforinthePBWM,andonneuralsubstrates outside of the PBWM’s BG-thalamus-PFC pathways.
Modellingtheseprocessesandtheirneuralbasisisnec- essarytoachieveacompleteneurocomputationalunder- standingofWM.
This review discusses recent progress toward this goal.
Wefocus on recent effortsto linkbrain measurements with behaviour on the reference-back task (Figure 2;
[24,21]),a WM-based decision-makingtask thatpro- videsseparatebehaviouralmeasuresofgateopeningand closing,aswellasupdatingandsubstitutionprocessesnot accountedforinthePBWM.Indoingso,wesuggestthat
Figure1
Posterior Cortex Frontal Cortex
STN
GPe
SNr NoGo
Go thalamus
VA,VL,MD
excitatory inhibitory
dorsal striatum
D1 D2
Current Opinion in Behavioral Sciences
IllustrationofthePBWMmodel.Gateopeningiscontrolledbya striatal‘go’signalthatinhibitsSNranddisinhibitsthalamusandPFC, enablingupdatingtooccur.Gateclosingiscontrolledbyastriatal‘no- go’signalthatinhibitsGPe,disinhibitsSNr,whichinhibitsthalamus andPFC,preventingupdating.Extendingthismodeltoinclude additionalstructuresimplicatedinWMandcognitivecontrol(e.g.
hippocampus,ventraltegmentalarea,anteriorcingulatecortex)and theirroleinWMsubprocessesbeyondgateopening/closingisakey targetformodel-basedcognitiveneuroscience.AdaptedfromHazy etal.[26]withpermission.
5The PBWM model assumes that WM sits in the ‘gate-closed’/
maintenancemodebydefault.Wenotethatthisassumptionislikely toostrong,sinceitimpliesthatgateopeningmustalwaysaccompany updating.UnderthisassumptionthePBWMwouldfailtopredictthe differentgatingcoststoWMupdatingthatoccurinbehaviouraldata(e.
g.Refs.[24,21]).
6ThePBWMmodelsuggestsaphasicdopaminergicsignalfromthe midbraindopaminestructuresonlyintheearlyphasesofaWMtask whentheBGmustlearnwhentoupdate.OnceWMupdatingrulesare learned,BGnucleinolongerrelyonaphasicdopaminergicresponsebut control WM gating via the non-dopaminergic SNr. Any additional dopaminergicinputreflects eitherreward associationsora feedback- based response which evaluates theupdating process based on the rewardpredictionerrorcodedbythesameneurons[84].Thisresponse, intheformof burstsanddips indopaminergic releaseontostriatal neurons,isthoughttoreinforce‘go’and‘no-go’activation,respectively.
furtherprogresscanbemadebyapplyingthemethodsof model-basedcognitiveneuroscience[41,42],whichlinksbrain activityto behaviour viadetailedcomputationalmodels of cognitive and neuralprocesses[43–45].Model-based cognitive neuroscience generates detailed quantitative theories that span multiple levels of abstraction (e.g.
behavioural, cognitive, neural). This provides greater constraint on theory and leads to more robust and detailed inferences. In particular, combining model- based approacheswith developmentsin ultra-high field fMRI enables testing neurocomputational theories of WM (such as thePBWM)with greaterspatial andpsy- chometric precision than has previously been possible.
Applyingthesemethodstothereference-backtaskpro- misesamoredetailedneurocomputationalunderstanding of WMthaniscurrently available.
Measuring WM subprocesseswith the reference-back paradigm
Most laboratory tasks used to study WM (e.g. n-back, delayed-match-to-sample)aredesignedtoinvestigatethe capacityandtemporalpropertiesofWMbutareunableto differentiate the contribution of WM subprocesses to observedbehaviour([24,46,47,48,21,22]).Arecently developed exception isthe reference-backtask[24,21], which provides dissociable measures of core WM sub- processes(gateopening,gateclosing,updating,substitu- tion)frombehaviouralchoice-response time(RT)data.
Toperformthereference-back, participantsholdoneof two stimuli (e.g. an ‘X’ or ‘O’) in WM while deciding whetheraseriesofprobesmatchthecurrentiteminWM
(Figure 2). On reference trials (indicated by ared frame around thestimulus), theparticipant must updateWM with the currently displayed stimulus. On comparison trials(indicatedbyablueframe),theparticipantsimply comparesthecurrentstimulustotheoneheldinWM(the one appearing in the most recent red frame) without updating WM. Both reference and comparison trials requireasame/differentdecisionbutonlyreferencetrials require updating. Comparingperformanceon reference and comparison trials thus provides abehavioural mea- sure of the cost of updating.By similar logic, switching fromcomparisontoreferencetrialsrequiresopeningthe WM gate (to allow for updating), whileswitching from reference to comparison trials requires closing theWM gate (to maintain the current contents). Gate opening is measured by comparing trials on which participants switchtowardsareferencetrialtothosewherereference trialsarerepeated. Likewise,gateclosingismeasured by comparingtrials onwhich participantsswitchtowards a comparison trial to those where comparison trials are repeated.Finally,substitutionismeasuredviatheinterac- tioneffectoftrialtype(reference/comparison)andmatch type(same/different)andrepresentsthecostofupdating anewitem intoWM.
Thebenchmarkbehaviouralfindingfromthereference- backtaskisthattrialsrequiringadditionalWMprocesses tend to have slower RTs and/or more frequent errors than trials that do not require such processes [24,47,49,21,50,51,52]. These costs are typically interpreted asreflectingacombinationoftime required for additional subprocesses to run outside of the same/
Figure2
Trial type:
Switching:
Matching:
Response:
X O O X O X X
reference reference reference reference
mismatch mismatch mismatch mismatch match match
‘different’ ‘different’ ‘different’ ‘different’ ‘same’ ‘same’
comparison comparison comparison
no-switch no-switch
-
- -
switch (gate closing)
switch (gate closing) switch
(gate opening)
switch (gate opening)
Current Opinion in Behavioral Sciences
Illustrationofthereference-backtask.Oneachtrial,participantsindicatewhetherthepresentedletterissameordifferentfromtheletterinthe mostrecentredframe.Onreference(redframe)trials,participantsmustalsoupdateWMwiththecurrentlydisplayedletter.Oncomparison(blue frame)trials,participantsmakethesame/differentdecisionbutdonotupdateWM.Comparingbehaviouraloutcomes(e.g.responsetime,error rate)betweendifferenttrialtypesmeasuresthecostofgateopening,gateclosing,updating,anditemsubstitutionprocesses(seetextfordetails).
Explainingthesebehaviouralphenomenaviacomputationalcognitivemodelsandestablishingfurtherlinkstoneuraldataisakeygoalofcurrent WMresearch.AdaptedfromRac-LubashevskyandKessler[21]withpermission.
different decision stage, and subprocesses interfering withtheprimary task(e.g. creatingnoisierWMrepre- sentationsduetodrawingattention/capacityawayfrom the decision process) [53]. However, distinguishing these accounts requires detailed choice-RT models of the latent cognitive processes underlying memory- based decision making(e.g. the highly successful evi- denceaccumulationframework,[54,55]),whichareyetto be applied to the reference-back paradigm. Before discussingapproachestomodellingthereference-back task,wefirstreviewrecenteffortstoidentifytheneural substrates of WM subprocesses by correlating brain activity with behavioural measures derived from the reference-back.
Neural correlatesofthe reference-backtask Asoutlinedabove,thereisbroadconsensusfromneuro- imagingsupportingtheroleofBG,thalamus,andPFCin WMgatingasinstantiatedinthePBWM[27].However, theneural basis of several coreWM subprocesses (e.g.
gateclosing,updating,substitution)islessclear.Recent workhasbeguntoaddressthisgapbylinkingbehavioural measures derived from the reference-back with neuro- physiological measures such as EEG and fMRI [47,49,51,52].
Two initial studies investigated EEG correlates of the reference-backtask.Rac-LubashevskyandKessler[51] found that gate closing was associated with increased theta power, a neural signature of cognitive control [56–58],whilegateopeningandupdatingwereassociated withincreaseddeltapower,asignatureofreactive(event- driven)controlandactionselectionprocessesthatengage in response to reward prediction errors [59–61]. This suggests afunctional role for deltaand theta signals in thecontrolofWMconsistentwith‘go/no-go’signallingin thePBWMmodel[25,39,26].Afollow-upstudyexplored theroleoftheP3bEEGsignal(apositiveevent-related potential that signals task-relevant events and peaks 300ms after stimulus onset) in gating and updating [52].P3b amplitudespikeddepending onwhetherthe stimulus matched the WM reference item, implicating P3b in stimulus comparison/categorisation processes rather than updating per se. Greater negative activity (in an N2-like ERP component unrelated to the P3b) wasfoundinanteriorcorticalregionsonreferenceversus comparison trials. This signal has been implicated in controlled inhibition and action selection [62] and, in thecontext of thereference-back task, likely reflectsa gate-opening or updating signal, consistent with the PBWM’s assumption that reference trials trigger an update or ‘go’ signal to allow new information into WM. This initial work demonstrates that neural signa- turesofspecificupdatingandgatingprocessesaredetect- ableinEEGoscillatorysignalsthatshowactivitybroadly consistent with ‘go/no-go’ signalling in BG-thalamus- PFCpathwaysinvolvedinWMgating[25,26].However,
thepoor spatial resolution of EEG limits our ability to drawconclusionsaboutthespecificstructuresassociated witheachWMsubprocess.
Extending this work, Nir-Cohen et al. [47] used 3T fMRItoidentifyneuralsubstratesofWMsubprocesses using a modified reference-back with more complex face-morph stimuli. BG, frontoparietal cortex, and task-relevant sensory areas such as visual cortex were involvedingateopening.Gateclosingactivatedparietal cortexand substitution elicitedactivationinleft dorso- lateralPFC and inferior parietal lobule. Awhole-brain conjunction analysis revealed shared activity in the supplementarymotorareaforupdatingandsubstitution, while updating and gatingboth activated the posterior parietal cortex. These results broadly agree with the PBWM model [26] and support the role of BG and PFC in controlling the flow of information into WM andreplacingoldwithnewinformation.However,pari- etalcortexactivationduringgateclosingisnotpredicted by the PBWM. This suggests that additional brain structuresareinvolvedin controllingWMsubprocesses and points to an opportunity to extend the PBWM to explain the neural basis of WM subprocesses beyond gateopening.
Jongkees[49] provided furtherevidence for thedopa- minergicbasisofWMgatingandupdating processesby administering dopamine precursor L-tyrosine to young adults and comparing reference-back performance to a placebo-control group. The L-tyrosine group had less variablegate opening times thanplacebo controls,sug- gestingthatthedrugimprovedWMperformanceforpoor performersbut impairedhighperformers.Therewasno effectonupdatingorgateclosing,consistentwiththerole ofstriataldopaminesignalsinopeningthegatetoWMin linewiththePBWM[25,26].Furtherindirectsupportfor striataldopamineinvolvementcomesfromastudylink- ingevent-basedeye-blinkrate(aproxymeasureofstriatal dopamine) to WM updating in the reference-backtask [50].However,follow-upworkcombiningthisapproach with ultra-high field fMRI is needed to identify how activity in small subcortical structures as well as layers incortex(e.g.striatum,GP,thalamus,PFC)ismodulated bydopamine.
Currentdirections
The work reviewed above has taken important first stepstowardidentifyingtheneuralsubstratesofWM subprocesses beyond the BG-thalamus-PFC ‘go/no- go’gatingmechanismofthePBWM[39,26].However, existingworkhassofarbeenlimitedtorelatingbrain activity directly to the reference-back’s behavioural measures rather than the latent cognitive processes that give rise to behaviour. Model-based approaches that link brain and behaviour via computational cognitive models offer numerous advantages over
traditional statistical analyses of mean RT and error rateinunderstandingthecognitiveandneuralbasisof WM. For example, applying evidence accumulation modelsofchoice-RT(e.g.Refs.[54,55])toreference- back data would reveal whether performance costs occur because WMsubprocesses add time outside of thedecisionstage(longernondecisiontime),interfere with the decision process itself (reduced or noisier processingrate; [53]),orinducestrategic adjustments engaging top-down cognitive control (increased response caution). Decomposing behavioural effects (e.g.gating, updatingcosts)into asetoflatentcogni- tive processes (e.g. accumulation rate, nondecision time, cognitive control of thresholds) rather than coarse behavioural-level summary statistics enables exploringtheneuralsubstratesofWMingreaterdetail thanispossiblewithtraditionalmethods[63,64].This places stronger constraints on theory and ultimately produces more robust and detailed inferences about thelatentprocessesthatgeneratebehaviour.Applying cognitive models to the reference-back holds great promise in this regard.
Initsstandardform,thereference-backparadigmignores several important additional WM processes. These includemechanismsthatoperateoninformationalready active in WM [65–67], such as object selection and retrieval [7], item-specific removal ([23]; but see Ref.
[68], for evidence of removal in the reference-back), and groupingand reorganizationoperations(e.g. sorting itemsintoalphabeticalor chronologicalorder,chunking or grouping items together to form a single accessible representation, changing the serial position of items;
[69–72]). These mechanisms support effective remem- beringbyrestructuringinformationintomorememorable formats andensuringonlyrelevant informationismain- tainedandretrievedfromWM.Thestandardreference- backalsoignoresphenomenaassociatedwithWM’slim- ited capacity (e.g. WM load/set-size effects; [73–75,7]) and thetemporaldegradation(e.g.bydecayorinterfer- ence)ofWMrepresentations(forareview,seeRef.[76]).
Analyses that do not account for these processes risk misattributing theireffects to other processes, resulting in biasedinferences.
Simpleextensionstothereference-backtask(e.g.using multiple-item WM sets, inserting delays between the updatecue andstimuluspresentation),however,enable testing such effects alongside the gating and updating processes of the standardreference-back. For example, Verschooren etal. [77]developed amodified reference- back paradigmwhereone among severalitems inlong- termmemoryorperceptionisgatedintoWM.Thisallows forcomparinggatingdynamicsforperceptualversuslong- termmemoryinformation.Similarmultiple-itemmodifi- cations can be used to investigate some of the WM phenomena described above, including informing the
ongoing debate about whether items in WM are held inasmallnumberofdiscretehigh-precisionslots[74]or allocated capacity from a limited pool of continuous resources [78–80]. In discrete slots models, the fidelity ofitemsinWMonlydegradesonceallmemoryslotsare full(e.g.whenn>4).Incontinuousresourcemodels,an item’sfidelityisdeterminedbyitsshareoftheavailable resourcesandthusshoulddegradeininverseproportion tothetotalnumberofitemsinWM7.Evidenceaccumu- lation models are well suited to test between these competing accounts(e.g. viaaccumulation rateparame- ters) as they can beused to assess the fidelity of WM representations and measure capacity-sharing effects;
[74,81]).Varyingsetsizeinthereference-backandasses- singtheeffectsondecision-makingandWMprocesses(as measuredbycognitivemodels)couldtestbetweenslots andresourcearchitectures.Similarly,combiningamulti- ple-itemreference-backtaskwithreinforcementlearning (e.g.byreinforcingsomeitemsbutnotothers)couldshed light on the interplay between WM and learning (e.g.
Refs. [73,75]) and the role of expected value in WM- baseddecisions.Overall,webelievethatdetailedchoice- RT modelling will play an important role in resolving these important questions and in explaining additional WM phenomenacaptured byvariants ofthe reference- backtask.
Combiningcomputationalapproacheswithrecentdevel- opments in ultra-high field fMRI (7T and higher)(e.g.
increased resolution and better signal- and contrast-to- noiseratios)holdsgreatpromiseforidentifyingactivityin small subcortical structures (e.g. GP, SN, subthalamic nucleus,VTA;[82,83])andgainingadeeperunderstand- ing of their functional role in WM than is currently available.Forexample,thiswouldenableastrongertest of the so-called ‘third phase’ response of the PBWM model [27], which evaluates the updating process via dopaminergicmidbrainneuronsthatcoderewardpredic- tion errors[84].UnderthePBWM, midbraindopamine responses thattrain the BG whento update should no longeroccuronceupdating-relatedtaskruleshavebeen learned. This mechanism has proven difficult to verify with low fieldstrength fMRI[85,86],however,imaging reference-back performance with ultra-high field fMRI and linking neural measurements to cognitive model parameters would enable identifying these anatomical andfunctionalmechanismsingreaterdetailandprovide additional constraint oncognitive modelsof WM. Spe- cifically,whenmodellingtwoormoresourcesofdata(e.g.
fMRI and choice-RT) simultaneously, the power to detect joint effects (e.g. correlations between BOLD
7Note,however,thatcontinuousresourcemodelscanmimicdiscrete slotsmodels.Forexample,ifaresourcepoolhascapacitytoaccommo- date fouritems, thenitem fidelitymay onlybegintodegrade once demands exceed capacity (i.e. when n>4), thus producing similar predictions toadiscreteslotsmodel.Carefulexperimental designis neededinordertocorrectlyattributeeffectstocapacitylimitations[99].
signalandcognitivemodelparameters)isdeterminedby thesignal-to-noise ratiosofeachdatasource.Increasing thesignal-to-noiseratioofneuraldata(e.g.via7TfMRI;
[82])reducesuncertaintythroughoutthemodel,asdoes including data from additional modalities (e.g.
EEG+fMRI+behavioural; [87])8. A further benefit is thatconnectingneuralsignalstocognitivemodelparam- etersallowsforselectingbetweencognitivemodelsthat makeidenticalpredictionsatthelevelofchoice-RTbut differ in their internal dynamics [45,64,88,89].That is, differentinternalmechanismscanbetitratedbyevaluat- ingwhichismostconsistentwiththeadditionalstructure providedbytheneuraldata.Combiningsuchapproaches withthereference-backtaskhaspotentialtoshedlighton other structures known to be involved in WM (e.g.
hippocampus;[90,91,92]),dopaminergicresponseevalu- ation (e.g. VTA; [93,94]), and cognitive control (e.g.
anterior cingulate cortex; [95]), which are not yet accounted for in existing neurocomputational models.
Linking state-of-the-art fMRI to the latent cognitive processes engaged by the reference-back would offer particular insight into the function of small dopamine- producing midbrain structures, with implications for understanding WM impairments in a range of clinical disorders involving abnormal dopamine function [96].
Overall,webelievethatviewingthereference-backtask throughthelensof model-basedcognitiveneuroscience promises a more detailedunderstanding of thesubpro- cessesthatsupportWMandtheirneuralsubstrates.
Concludingremarks
Thisreviewdiscussedrecenteffortstoidentifytheneural basis of subprocesses that support WM in therecently developed reference-back task. Current empirical work supportstheideathatWMgatingiscontrolledbystriatal
‘go/no-go’ signalling in BG-thalamus-PFC pathways.
However, the neural substrates of several additional WMsubprocesses areyetto beestablished,pointing to aneedfor ultra-highfield functionalimagingcombined withdetailedcomputationalcognitivemodelling.Targets forfutureresearchincludeextendingthereference-back task to account for additional WM subprocesses (e.g.
removal, selection, and reorganization operations) and effects of WM load and capacity (e.g. longer retrieval times, noisier WM representations), as ignoring such processesleads to mis-specifiedmodels and potentially biasedinferences.Applyingthemethodsofmodel-based cognitiveneurosciencetothereference-backtaskwould provideamajoradvanceinunderstandingWMatneural, cognitive, and behavioural levels. A comprehensive understandingofWMsubprocessesandtheirneuralbasis iswithinreach,withimplicationsforbothcognitiveand clinicalneuroscience.
Authorcontributions
RJBandACTconductedtheliteraturereviewandledthe writingofthemanuscript.Allauthorsprovidedfeedback at different stages, reviewed, edited, and revised the manuscript.
Conflictofintereststatement Nothingdeclared.
Acknowledgements
WethankDrYoavKesslerformakingseveralsuggestionsthatimprovedan earlierversionofthismanuscript.Thisworkwassupportedbyagrantfrom theNetherlandsOrganisationforScientificResearch(NWO;grantnumber 016.Vici.185.052;BUF).
Referencesand recommendedreading
Papersofparticularinterest,publishedwithintheperiodofreview, havebeenhighlightedas:
ofspecialinterest ofoutstandinginterest
1. CowanN:Evolvingconceptionsofmemorystorage,selective attention,andtheirmutualconstraintswithinthehuman information-processingsystem.PsycholBull1988,104:163.
2. DanemanM,CarpenterPA:Individualdifferencesinworking memoryandreading.JMemLang1980,19:450.
3. KyllonenPC,ChristalRE:Reasoningabilityis(littlemorethan) working-memorycapacity?! Intelligence1990,14:389-433.
4. CowanN:Themagicalnumber4inshort-termmemory:a reconsiderationofmentalstoragecapacity.BehavBrainSci 2001,24:87-114.
5. GaravanH:Serialattentionwithinworkingmemory.MemCogn 1998,26:263-276.
6. OberauerK:Accesstoinformationinworkingmemory:
exploringthefocusofattention.JExpPsycholLearnMem Cognit2002,28:411.
7. SewellDK,LilburnSD,SmithPL:Objectselectioncostsinvisual workingmemory:adiffusionmodelanalysisofthefocusof attention.JExpPsycholLearnMemCognit2014,42:1673.
8. DreisbachG:Mechanismsofcognitivecontrol:thefunctional roleoftaskrules.CurDirPsycholSci2012,21:227-231.
9. DreisbachG,Fro¨berK:Onhowtobeflexible(ornot):
modulationofthestability-flexibilitybalance.CurrDirPsychol Sci2019,28:3-9.
10. HommelB:Betweenpersistenceandflexibility:theYinand Yangofactioncontrol.AdvancesinMotivationScience.Elsevier;
2015:33-67.
11. DreisbachG,Mu¨llerJ,GoschkeT,StrobelA,SchulzeK,LeschKP, BrockeB:Dopamineandcognitivecontrol:theinfluenceof spontaneouseyeblinkrateanddopaminegene
polymorphismsonperseverationanddistractibility.Behav Neurosci2005,119:483.
12. MusslickS,JangSJ,ShvartsmanM,ShenhavA,CohenJD:
Constraintsassociatedwithcognitivecontrolandthe stability-flexibilitydilemma.Proc.40thAnnualMeetingofthe CognitiveScienceSociety;CognitiveScienceSociety:Madison, WI:2018:804-809.
13.
CoolsR:Chemistryoftheadaptivemind:lessonsfrom dopamine.Neuron2019,104:113-131.
Reviewsthedopaminergicbasisofthestability-flexibilitytrade-offfaced bycognitivesystems.Providesanoutlineofchemicalneuromodulatory mechanismsandtheirroleinenablingflexiblegoal-directedcognition.
14. CoolsR,D’EspositoM:Inverted-U-shapeddopamineactions onhumanworkingmemoryandcognitivecontrol.Biol Psychiatry2011,69:e113-e125.
8Thisisparticularly importantforindividual differencesanalyses, whichrelyonprecisemeasurementattheindividualleveltoaccurately capturethevariationbetweenpeople.
15. BadreD:Openingthegatetoworkingmemory.ProcNatlAcad SciUSA2012,109:19878-19879.
16. BledowskiC,KaiserJ,RahmB:Basicoperationsinworking memory:contributionsfromfunctionalimagingstudies.
BehavlBrainRes2010,214:172-179.
17. KesslerY,OberauerK:Workingmemoryupdatinglatency reflectsthecostofswitchingbetweenmaintenanceand updatingmodesofoperation.JExpPsycholLearnMemCognit 2014,40:738.
18. MillerEK,CohenJD:Anintegrativetheoryofprefrontalcortex function.AnnuRevNeurosci2001,24:167-202.
19. MurtyVP,SambataroF,RadulescuE,AltamuraM,IudicelloJ, ZoltickB,WeinbergerDR,GoldbergTE,MattayVS:Selective updating of working memory content modulates meso-cortico-striatalactivity.Neuroimage2011,57:1264- 1272.
20. O’ReillyRC:Biologicallybasedcomputationalmodelsofhigh- levelcognition.Science2006,314:91-94.
21.
Rac-LubashevskyR,KesslerY:Decomposingthen-backtask:
anindividualdifferencesstudyusingthereference-back paradigm.Neuropsychologia2016,90:190-199.
ExploredindividualdifferencesinWMsubprocessesmeasuredbythe reference-backtaskandcomparedreference-backperformancetosev- eral variants ofcommonly usedWM tasks. Thereference-back task provided measuresofhowgateopening,gateclosing, andupdating eachcontributetoindividualtaskperformance.Controllingthecontents ofWMisshowntobeaprimarysourceofindividualdifferencesinWM- baseddecisionmaking.
22. RothJK,SerencesJT,CourtneySM:Neuralsystemfor controllingthecontentsofobjectworkingmemoryinhumans.
CerebCortex2006,16:1595-1603.
23. EckerUK,OberauerK,LewandowskyS:Workingmemory updatinginvolvesitem-specificremoval.JMemLang2014, 74:1-15.
24.
Rac-LubashevskyR,KesslerY:Dissociatingworkingmemory updatingandautomaticupdating:thereference-back paradigm.JExpPsycholLearnMemCognit2016,42:951.
Introducedthereference-backparadigm.Theexperimentsinthisstudy providedinitialbehaviouralevidenceforagatingmechanismthatsepa- ratesWMfromlong-termmemory.
25. FrankMJ,LoughryB,O’ReillyRC:Interactionsbetweenfrontal cortexandbasalgangliainworkingmemory:acomputational model.CognAffectBehavNeurosci2001,1:137-160.
26. HazyTE,FrankMJ,O’ReillyRC:Banishingthehomunculus:
makingworkingmemorywork.Neuroscience2006,139:105- 118.
27. O’ReillyRC,FrankMJ:Makingworkingmemorywork:a computationalmodeloflearningintheprefrontalcortexand basalganglia.NeuralComput2006,18:283-328.
28. CoolsR,SheridanM,JacobsE,D’EspositoM:Impulsive personalitypredictsdopamine-dependentchangesin frontostriatalactivityduringcomponentprocessesofworking memory.JNeurosci2007,27:5506-5514.
29. FrankMJ,O’ReillyRC:Amechanisticaccountofstriatal dopaminefunctioninhumancognition:
psychopharmacologicalstudieswithcabergolineand haloperidol.BehavNeurosci2006,120:497-517http://dx.doi.org/
10.1037/0735-7044.120.3.497.
30. JochamG,KleinTA,UllspergerM:Dopamine-mediated reinforcementlearningsignalsinthestriatumand
ventromedialprefrontalcortexunderlievalue-basedchoices.
JNeurosci2011,31:1606-1613.
31. DahlinE,NeelyAS,LarssonA,Ba¨ckmanL,NybergL:Transferof learningafterupdatingtrainingmediatedbythestriatum.
Science2008,320:1510-1512.
32. LewisSJ,DoveA,RobbinsTW,BarkerRA,OwenAM:Striatal contributionstoworkingmemory:afunctionalmagnetic resonanceimagingstudyinhumans.EurJNeurosci2004, 19:755-760.
33. McNabF,KlingbergT:Prefrontalcortexandbasalganglia controlaccesstoworkingmemory.NatNeurosci2008,11:103- 107.
34. TanHY,ChenQ,GoldbergTE,MattayVS,Meyer-LindenbergA, WeinbergerDR,CallicottJH:Catechol-O-methyltransferase Val158Metmodulationofprefrontal–parietal–striatalbrain systemsduringarithmeticandtemporaltransformationsin workingmemory.JNeurosci2007,27:13393-13401.
35. vanSchouwenburgMR,denOudenHE,CoolsR:Thehuman basalgangliamodulatefrontal-posteriorconnectivityduring attentionshifting.JNeurosci2010,30:9910-9918.
36. FeredoesE,HeinenK,WeiskopfN,RuffC,DriverJ:Causal evidenceforfrontalinvolvementinmemorytarget maintenancebyposteriorbrainareasduringdistracter interferenceofvisualworkingmemory.ProcNatlAcadSciUS A2011,108:17510-17515.
37. D’EspositoM,PostleBR:Thecognitiveneuroscienceof workingmemory.AnnuRevPsychol2015,66:115-142.
38. OwenAM,McMillanKM,LairdAR,BullmoreE:N-backworking memoryparadigm:ameta-analysisofnormativefunctional neuroimagingstudies.HumBrainMapp2005,25:46-59.
39. ChathamCH,BadreD:Multiplegatesonworkingmemory.Curr OpinBehavSci2015,1:23-31.
40. BraverTS:Thevariablenatureofcognitivecontrol:adual mechanismsframework.TrendsCognSci2012,16:106-113.
41. ForstmannBU,WagenmakersEJ(Eds):AnIntroductiontoModel- basedCognitiveNeuroscience.NewYork:Springer;2015:139- 156.
42. TurnerBM,PalestroJJ,MileticS,ForstmannBU:Advancesin techniquesforimposingreciprocityinbrain-behavior relations.NeurosciBiobehavRev2019,102:327-336.
Summarises recent advancements in approaches to simultaneously modelling brain-behaviour relationships using the joint modelling approachcentraltomodel-basedcognitiveneuroscience.Multipleneural datastreams (e.g.EEG, fMRI)canbeusedtoinformlatentcognitive modelparameters,thusprovidingabettermechanisticunderstandingof theneuralbasisoflatentcognitivemechanismsthatgiverisetoobserved behaviour.
43. FristonKJ:Modalities,modes,andmodelsinfunctional neuroimaging.Science2009,326:399-403.
44. LoveBC:Cognitivemodelsasbridgebetweenbrainand behavior.TrendsCognSci2016,20:247-248.
45. SchallJD:Accumulators,neurons,andresponsetime.Trends Neurosci2019.
46. EckerUK,LewandowskyS,OberauerK,CheeAE:The componentsofworkingmemoryupdating:anexperimental decompositionandindividualdifferences.JExplPsycholLearn MemCogn2010,36:170.
47. Nir-CohenG,KesslerY,EgnerT:Distinctneuralsubstratesfor openingandclosingthegatefromperceptiontoworking memory.bioRxiv2019http://dx.doi.org/10.1101/853630.
Used3TfMRItoidentifyneuralsubstratesofWMsubprocessesusingthe reference-backparadigm.Thisworkidentifiesbrainregionsinvolvedin gateopening,gateclosing,updating,andsubstitutionprocesses,broadly consistentwiththePBWMmodelinwhichBG-thalamus-PFCpathways controltheflowofinformationintoandoutofWM.
48. Lewis-PeacockJA,KesslerY,OberauerK:Theremovalof informationfromworkingmemory.AnnNYAcadSci2018, 1424:33-44.
Reviewsbehaviouralandneuralevidenceforaselectiveremovalprocess thatisengagedtoremoveoutdatedinformationfromWM.Theremoval mechanismservestolimitworkingmemoryloadbyensuringonlygoal- relevantinformationismaintainedinWM.Thewaysinwhichselective removal is distinct from temporal decay and interference are also discussed.
49. JongkeesBJ:Baseline-dependenteffectofdopamine’s precursorL-tyrosineonworkingmemorygatingbutnot updating.CognAffectBehavNeurosci2020:1-15.
InvestigatedthedopaminergicbasisofWMgatingandupdatingpro- cessesbyadministeringdopamineprecursorL-tyrosinetoyoungadults
