Évaluation de la charge cognitive lors de l'apprentissage pour un support tutoriel approprié

Evaluation de la charge cognitive lors de l'apprentissage pour un support

tutoriel approprie

par

Francis Courtemanche

memoire presente au Departement d'informatique

en vue de l'obtention du grade de maitre es sciences (M.Sc.)

FACULTE DES SCIENCES

UNIVERSITE DE SHERBROOKE

1*1

Library and

Archives Canada

Published Heritage

Branch

Bibliotheque et

Archives Canada

Direction du

Patrimoine de I'edition

Your file Votre reference ISBN: 978-0-494-49483-7 Our file Notre reference ISBN: 978-0-494-49483-7

NOTICE:

The author has granted a

non-exclusive license allowing Library

and Archives Canada to reproduce,

publish, archive, preserve, conserve,

communicate to the public by

telecommunication or on the Internet,

loan, distribute and sell theses

worldwide, for commercial or

non-commercial purposes, in microform,

paper, electronic and/or any other

formats.

AVIS:

L'auteur a accorde une licence non exclusive

permettant a la Bibliotheque et Archives

Canada de reproduire, publier, archiver,

sauvegarder, conserver, transmettre au public

par telecommunication ou par Plntemet, prefer,

distribuer et vendre des theses partout dans

le monde, a des fins commerciales ou autres,

sur support microforme, papier, electronique

et/ou autres formats.

The author retains copyright

ownership and moral rights in

this thesis. Neither the thesis

nor substantial extracts from it

may be printed or otherwise

reproduced without the author's

permission.

L'auteur conserve la propriete du droit d'auteur

et des droits moraux qui protege cette these.

Ni la these ni des extraits substantiels de

celle-ci ne doivent etre imprimes ou autrement

reproduits sans son autorisation.

In compliance with the Canadian

Privacy Act some supporting

forms may have been removed

from this thesis.

Conformement a la loi canadienne

sur la protection de la vie privee,

quelques formulaires secondaires

ont ete enleves de cette these.

While these forms may be included

in the document page count,

their removal does not represent

any loss of content from the

thesis.

Canada

Bien que ces formulaires

aient inclus dans la pagination,

il n'y aura aucun contenu manquant.

Lel0decembre2008

lejury a accepte le memoire de M. Frangois Courtemanche dans sa version finale.

Membres dujury

M. Andre Mayers

Directeur

Departement d'informatique

M. Yves Bouchard

Membre

- Faculte de theologie, d'ethique et de philosophic

Mme Helene Pigot

President-rapporteur

Departement d'informatique

Sommaire

Ce memoire aborde deux themes faisant partie du domaine des systemes tutoriels intelligents.

Un modele de l'apprenant reposant sur des theories psychologiques de l'apprentissage et du

traitement de 1'information sera d'abord presente, suivi de deux strategies pedagogiques tirant

profit de ce modele.

L'objectif general du projet de recherche est d'enrichir, a l'aide d'un modele cognitif,

P information disponible pour un systeme tutoriel dans le but de promouvoir plus

efficacement Pacquisition de connaissances lors d'activites d'apprentissage. Un laboratoire

de soustractions en base hexadecimale a ete developpe dans le but de tester et de valider

Papproche proposee.

Un article ayant fait Pobjet d'une publication et detaillant une partie du projet de recherche

est presente dans chacun des trois chapitres de ce memoire.

Remerciements

En preambule a ce memoire, je souhaite remercier toutes les personnes qui ont contribue a la

realisation de mon projet de maitrise par leur aide et leur soutien.

Mes premiers remerciements vont a Andre Mayers, mon directeur de recherche, pour les

nombreux conseils d'ordre theorique et methodologique ainsi que sa grande disponibilite. Je

lui dois la decouverte du fascinant domaine que sont les sciences cognitives

computationnelles. Un domaine qui me reserve encore bien des annees de plaisir.

J'adresse aussi de sinceres remerciements a Mehdi Najjar, que j'ai rencontre lors de ses

etudes postdoctorales a PUniversite de Sherbrooke. Au cours de nos frequentes

collaborations, j'ai pu beneficier de sa grande experience en recherche et en redaction

scientifique. Ses nombreux conseils m'ont ete et me sont encore d'une aide inestimable.

Enfin, je remercie tous les etudiants du groupe de recherche ASTUS. Particulierement

Jean-Francois Lebeau et Mikael Fortin qui m'ont apporte une aide appreciable sur plusieurs

questions de programmation.

Table des matieres

Sommaire ii

Remerciements ii

Table des matieres iv

Liste des abreviations vi

Liste des figures vii

Introduction 1

Problematique et methodologie 2

Chapitre 1 Modele de Papprenant 4

1.1 Evaluation de la charge cognitive 4

Chapitre 2 La boucle interne 22

2.1 Apprentissage supervise 23

Chapitre 3 La boucle externe 37

3.1 Choix du prochain probleme 37

Conclusion 42

Annexe A Mise en oeuvre 44

A.l Laboratoire de soustraction 44

A.2 Simulateur de memoire de travail 47

Annexe B Liste des packages Java 53

A.l Modele de l'apprenant 53

A.2 Simulateur de memoire de travail 53

A.2 Comportement du pedagogue 54

Liste des abreviations

ASTUS Apprentissage par Systeme Tutoriel de l'Universite de Sherbrooke

CLT Cognitive Load Theory

ECL Extraneous Cognitive Load

GCL Germane Cognitive Load

ICL Intrinsic Cognitive Load

IES Intelligent Educational System

ITS Intelligent Tutoring System

LO Learning Objective

LTM Long Term Memory

STI Systeme Tutoriel Intelligent

WM Working Memory

Liste des figures

Figure 1 - Laboratoire de soustraction hexadecimale 45

Figure 2 - Calculette de conversion 46

Figure 3 - Interface du tuteur 46

Figure 4 - Interface de visualisation, partie 1 48

Figure 5 - Interface de visualisation, partie 2 49

Figure 6 - Interface de visualisation, partie 3 51

Introduction

Parvenir a un niveau de qualite d'enseignement se rapprochant de plus en plus de celui que

Ton retrouve dans les vraies salles de classe a toujours ete l'ideal a atteindre dans le domaine

des systemes tutoriels intelligents. Malgre les nombreux succes des systemes tutoriels

intelligents (STI) depuis les debuts de leur creation comme domaine de recherche a part

entiere, la relation privilegiee entre l'enseignant et ses eleves demeurera toujours

irremplacable. Recreer entierement la richesse et la complexity de cette relation n'est

evidemment pas non plus l'objectif des chercheurs de cette discipline. Simplement faire un

pas de plus dans cette direction est deja en soi une reussite.

Durant les premieres annees de recherche sur les STI, beaucoup d'efforts ont ete investis pour

aborder la relation d'enseignement en partant du cote du tuteur. De la ont emerge des

strategies et des agents pedagogiques de plus en plus raffines et efficaces. Toutefois, depuis

les dernieres annees, la recherche tend a s'orienter dans l'autre direction. On aborde

maintenant la relation d'enseignement du cote de l'apprenant. Les chercheurs tentent de creer

des modeles representant certaines caracteristiques precises de l'etudiant pouvant etre

exploitees efficacement par un agent tuteur. On retrouve done dans la litterature de plus en

plus d'etudes theoriques et techniques portant sur les aspects emotifs et cognitifs intervenant

dans l'apprentissage. C'est dans ce courant que s'inscrivent les travaux de recherche

presentes dans ce memoire.

Problematique et Methodologie

De maniere generate, lors de la resolution de problemes par un etudiant, la performance se

degrade a mesure que la difficulte et l'effort mental augmentent. La theorie de la charge

cognitive a pour but de decrire plus formellement ce phenomene. Elle nous indique que lors

de seances d'apprentissage, l'acquisition des connaissances est soumise aux memes

contraintes [13]. Passe un certain seuil, le taux d'apprentissage diminue. Puisque les

systemes tutoriels intelligents developpes au sein du groupe de recherche ASTUS

(Apprentissage par Systeme Tutoriel de l'Universite de Sherbrooke) sont de type « resolution

de problemes », il devient alors essentiel de tenir compte des contraintes liees a l'effort

mental afin de maximiser l'apprentissage.

Pour resoudre cette problematique, le premier objectif est de modeliser les contraintes

cognitives liees a l'apprentissage en contexte de resolution de problemes. Un deuxieme

objectif consiste a elaborer des strategies pedagogiques tenant compte de ces contraintes et

des connaissances enseignees, afin de maintenir l'effort mental de 1'etudiant a un niveau

constant et optimal.

La methodologie mise de l'avant pour realiser ces objectifs se divise principalement selon les

quatre etapes suivantes:

1) Modeliser la memoire de travail d'un etudiant utilisant un laboratoire

virtuel d'apprentissage.

2) Mettre au point un domaine d'apprentissage qui mobilise de maniere

simple mais soutenue la memoire de travail de 1'etudiant.

3) Estimer de facon computationnelle la charge cognitive ressentie

durant la resolution de problemes.

4) Elaborer des strategies pedagogiques qui permettent au systeme

tutoriel de guider l'etudiant de maniere a optimiser son effort mental

et maintenir un niveau suffisant d'attention.

Le memoire est structure de la maniere suivante :

Au sein de la litterature dans le domaine des systemes tutoriels intelligents, on distingue

essentiellement quatre composants aux STI [15]: le module expert, le module pedagogique, le

modele de l'apprenant et le module de communication (agent interface). Le premier chapitre

du memoire est consacre a notre approche concernant le modele de l'apprenant. Nous y

exposons sa principale originalite: l'utilisation d'un simulateur de memoire de travail

permettant d'estimer l'effort mental investi par un apprenant lors de la resolution d'un

probleme.

On decrit generalement le comportement des systemes tutoriels selon deux grands axes ; la

boucle interne et la boucle externe [14]. La premiere est responsable de fournir les services

pedagogiques au cours des activites d'apprentissage, alors que la seconde a pour objectif de

selectionner les problemes a soumettre a l'apprenant. Le chapitre 2 du memoire concerne nos

travaux sur la boucle interne. Nous montrons comment l'utilisation de patrons cognitifs,

fournis par le modele de l'apprenant, permet l'elaboration d'une strategic d'apprentissage

supervise favorisant l'acquisition de connaissances.

Finalement, le chapitre 3 porte sur l'application de la boucle externe. La strategie de selection

de taches qui y est exposee permet de soumettre a l'etudiant une suite de problemes tenant

compte de ses besoins et de ses capacites. Cette approche personnalisee repose sur

l'estimation de la performance et de la charge cognitive de l'apprenant.

Chapitre 1

Modele de Papprenant

Dans le cadre des systemes tutoriels intelligents, le modele de l'apprenant est responsable

d'evaluer - de maniere implicite ou explicite - le comportement et les performances de

l'etudiant [13]. Que ce soit en colligeant les bonnes et les mauvaises reponses, en fournissant

de 1'information sur les habitudes generates d'apprentissage ou en evaluant l'effort mental de

l'etudiant, le modele de l'apprenant sert de base aux decisions concernant les strategies et les

interventions pedagogiques. L'article presente dans ce chapitre detaille l'ensemble de

l'approche developpee afm d'evaluer l'effort mental investi par un apprenant lors d'activites

de resolution de problemes, dans le but d'enrichir le modele de l'apprenant d'un point de vue

cognitif.

1.1 Evaluation de la charge cognitive

La strategie principale adoptee par le groupe ASTUS, dans le developpement de systemes

tutoriels, est de reifier les connaissances manipulees par l'apprenant via l'interface des

laboratoires d'apprentissage [7]. Pour chaque etape de resolution, il est done possible, dans

une certaine mesure, de determiner le contenu de la memoire de travail de l'apprenant. II

s'agit des connaissances qu'il manipule a un instant donne afin d'obtenir un certain resultat.

La theorie de la charge cognitive (cognitive load theory, ou CLT) est un cadre conceptuel issu

de la psychologie de l'apprentissage, qui formalise la relation entre les structures

d'information et les processus cognitifs humains [9]. L'objectif principal de cette theorie est

de definir des methodes d'instruction optimales, d'un point de vue cognitif, en tenant compte

de Pinformation communiquee a l'apprenant. Suivant l'approche d'explicitation des

connaissances utilisee dans ASTUS, la CLT offre un cadre ideal pour analyser 1'effort mental

investi par un apprenant en interaction avec un systeme tutoriel.

Cependant, la theorie de la charge cognitive repose essentiellement sur des methodes

subjectives pour determiner la charge cognitive engendree lors d'activite d'apprentissage

[10]. II s'agit par exemple de questionnaires administres a posteriori [11] ou de taches

secondaries disruptives imposees durant la resolution de problemes [2]. Ces methodes sont

intrusives et necessitent un delai pour la prise en compte de leurs resultats. L'avancement

actuel de la CLT impose done des contraintes qui entrent en conflit avec plusieurs des

objectifs des systemes tutoriels intelligents, soit une evaluation transparente de l'apprenant et

une action pedagogique en temps reel.

Un simulateur de memoire de travail a done ete developpe afin de surmonter ces obstacles et

de permettre une utilisation adequate de la theorie de la charge cognitive dans le contexte des

STL Le simulateur est une implementation du modele de Baddeley [12] et permet une

estimation en temps reel de la charge cognitive en prenant comme entrees les connaissances

manipulees dans l'interface des laboratoires.

L'article presente dans les pages suivantes developpe les points qui viennent d'etre introduits.

II a ete soumis a la revue Fundamenta Informatica pour un numero special portant sur

l'informatique cognitive. II est a noter qu'une version reduite {full paper) de cet article a ete

publiee a la 30

conference annuelle CogSci (Cognitive Science Society), qui s'est tenue du

23 au 26 juillet 2008 a Washington, D.C. aux £tats-Unis. II peut etre consulte aux pages

2304-2309 des actes de la conference.

Toward Computational Estimations of

Cognitive Loads

Francois Courtemanche

Department of Computer Science, University of Sherbrooke Sherbrooke, QC, Canada,

[email protected]

Mehdi Najjar

Interdisciplinary Research Center on Emerging Technologies, University of Montreal Montreal, QC, Canada,

[email protected]

Andre Mayers

Department of Computer Science, University of Sherbrooke Sherbrooke, QC, Canada,

[email protected]

Abstract. Understanding the cognitive constraints affecting learning is

primordial for the effectiveness of instructional design and intelligent learning system. This paper presents a simulator model for computational working memory load estimations during learning activities. Using a Cognitive Load Theory interpretation, the simulator estimations are used to predict learning outcomes while using a tutoring system.

1. Introduction

Nowadays, and thanks to many recent advances in computational technology, intelligent tutoring systems (ITS) are increasingly used in order to supplement real classroom teaching. Nevertheless, an important issue concerns the effective learning outcomes of these systems. A

growing body of research emphasises the fact that working memory constraints are determinant regarding instructional effectiveness (Clark, Nguyen & Sweller, 2006; Sweller, van MerrienBoer & Paas, 1998). Within the field of ITS, few efforts have been made to take into account the improvement of the cognitive aspects related to their usage. Thus, a systematic framework is needed to assess ITS effectiveness from a cognitive point of view. The Cognitive Load Theory (CLT) has been developed to evaluate the impact of the human cognitive architecture on instructional design (Clark et al., 2006; Paas, Renkl & Sweller, 2004). Although CLT researches have shown successful results using empirical data, analytical measures and computational models are needed to go a step further in formalising the theory (Paas, Tuovinen, Tabbers, & Van Gerven, 2003). This paper presents a working memory simulator designed to computationally estimate cognitive load. The remainder of the paper is organised as follow. Section 2 exposes the cognitive load theory framework. Section 3 and 4 detail the theoretical and computational models of the working memory simulator. Section 5 explains how the cognitive load estimations receive a cognitive CLT-based interpretation using cognitive load patterns. Section 6 describes the empirical technique used to calibrate the working memory simulator. Concluding remarks are given in section 7.

2. The Cognitive Load Theory

The Cognitive Load Theory (CLT) is a conceptual framework formalising the effect resulting from the performance of complex cognitive tasks during learning (Paas & Van Merrienboer, 1994, Paas, Renkl & Sweller, 2004; Sweller, van MerrienBoer & Paas 1998). In order to design optimal instructional methods, CLT uses interactions between information structures and the human cognitive system. On a theoretical ground, the theory relies on two main learning mechanisms: knowledge automation and schema acquisition. The main goal of CLT is to find how these two learning mechanisms can be fostered during learning activities, using a limited multi-modal working memory, in order to improve knowledge structure in the learner's long-term memory.

As reported by Paas, Tuovinen, Tabbers & Van Gerven (2003), cognitive load is defined by three components: intrinsic cognitive load, extraneous cognitive load and germane cognitive load. Intrinsic cognitive load represents the load imposed on working memory by the intrinsic nature of the knowledge to be learned. It is affected by the number of items to be addressed simultaneously in working memory and their interactivity. This number depends in turn on the knowledge aggregation degree (schemas structure) in long-term memory. Information stored in long-term memory is organised in the form of schemas in order to bind together elements of knowledge handled to achieve common goals. Extraneous cognitive load represents all form of load which is not directly devoted to knowledge manipulation (e.g. visual and auditory presentation, instructional complexity, interface manipulation). This load is not effective for learning and can be reduced by a better instructional design. Germane cognitive load represents the load resulting from learning processes (automation, schema acquisition). These three types of cognitive load are additives. Their sum may not exceed working memory capacity without severely impairing learning or causing a failure of the ongoing task.

The approach proposed in this paper aims to provide online analytical estimations of cognitive load within the context of intelligent tutoring systems. Using a mathematical model of working memory - implemented as a computational simulator - the intrinsic part of the cognitive load is

estimated for every learner's interactions with the learning environment. In order to implement different pedagogical strategies, these estimations are further incorporated in cognitive patterns which describe the learner's mental effort.

3. The Theoretical model

This section describes the working memory and knowledge representation models used to formalise the learners mental operations.

3.1. The knowledge representation

The knowledge representation model is implemented as a dynamic graph including three types of memory structure, each one corresponding to a particular type of knowledge. These structures are generally accepted in the computational cognitive modeling literature (Najjar & Mayers, 2007) and represent semantic knowledge, procedural knowledge and episodic knowledge.

According to Neely (1989), semantic knowledge represents concepts taken in a broad sense where it can be any category of objects. In our approach, instances of semantic knowledge contain different slots associated with a particular type (e.g. numeric, string or other concepts).

Procedural knowledge is used to satisfy needs without using the attentional system (Anderson, 1993). In our approach, procedures are subdivided in two main categories: primitive procedures and complex procedures. The former are seen as atomic actions on semantic knowledge and are reified in the learning environment as interactions of the learner via the laboratory interface. The latter represent mental operations and are executed by sequences of actions, which satisfy scripts of goals. Complex procedures results are calculated via the computation of functions which cause cognitive load. To formalise the concept of function, we took up the definition of Halford (1998), where a function is defined as a mental process over variables.

Goals are perceived as intentions of the student cognitive system (Najjar & Mayers, 2007; Newell, 1990) and are implemented as generic statements retrieved from the semantic memory (Najjar & Mayers, 2006).

Episodic memory retains details about our experiences and preserves temporal relationships (Tulving, 1983). In our approach, the episodes are based on instantiation of goals. The episodic knowledge is organized according to goals and the procedures needed to achieve them. More precisely, each pair of goal and procedure realisation is explicitly encoded in an episode and stored in the learner episodic memory.

3.2. The learning process

The two main learning processes defined within the cognitive load theory are knowledge automation and schema acquisition (Sweller, 1998). In this paper, we focus on the knowledge automation process, which is applied to the complex procedures functions.

According to the Instance Theory of Automatization (Logan, 1988; Logan, 2002), automation reflects a transition from performance based on an initial algorithm to performance based on memory retrieval. The theory suggests that subjects store and retrieve representation of each individual encounter with a stimulus. These representations are encoded as processing episodes (Logan, 1988). When the stimulus is encountered again in the context of the same goal, a race

between the algorithm and the retrieval process begins. The process finishing first controls the answer. Thus the transition is governed by statistical processes representing both racer speed-ups. With practice, the retrieval process tends to win as more episodes enter the race. The next equation (Logan, 1988) expresses the probability that the algorithm is used.

W

Probability = - (i)

[algo first] (Wg + n\A/J

The algorithm and memory retrieval processes can be viewed as Poisson processes with rates Wa and nWm - where Wa and Wm are respectively the mean time for the algorithm and the memory processes and n is the number of memory traces. As mentioned by Logan (1988), if the mean for the algorithm equals the mean for the memory process, the probability that the algorithm finishes first equals l/(n+l).

In our approach, each procedure realisation is encoded in an episode and stored in the episodic memory. The learner intention represented in the episode's goal is used to model the procedure stimulus and acts as the retrieval cue. Thus, the student episodic memory can be used to estimate the automation level of complex procedures and their related functions. We adapted the Logan equation in order to take into account the available attentional resources available during encoding. The following equation is used to estimate the functions automation level:

Func] (N + 1)

where N = X (G C Li x r° '5)

i = 1

where i stands for each previous encoding. The i05 parameter is an attenuation factor used in

order to increase the effect of recent episodes and to lower the effect of older ones. The GCL variable represents the attentional resources available during each / encoding. This variable is estimated using the germane cognitive load available while the learner executed the step (c.f. section 5).

3.3 The Working Memory Model

The working memory simulator implementation is based on the Baddeley model (Baddeley, 1986; Baddeley, 2003; Repovs & Baddeley, 2006). Figure 2 illustrates its four components

S * Centra! \ executive ) Visuospatial sketchpad Episodic buffer Phonological loop Visual semantics Episodic LTM -«—*- Language

Figure 2 - Multicomponent Model of Working Memory

The central executive coordinates the simultaneous task executions and oversees the other modules. This component is also responsible for current information processing. The phonological loop is a phonetic active memory whose function is to maintain active acoustic encoded information through articulatory rehearsal processes. The visuospatial sketchpad holds and manipulates visual and spatial information. The episodic buffer is a multidimensional store that enables the maintenance of integrated information. This buffer has the ability to create and manipulate novel representations and could be regarded as the storage component of the central executive. The maintenance of information within the episodic buffer depends on the central executive limited attentional capacity.

4. The Working Memory Simulator

Our computational implementation of the Baddeley working memory model contains the central executive, the episodic buffer and the visuospatial sketchpad. Because the approach described in this paper only concerns estimation of intrinsic cognitive load, the phonological loop is not incorporated. In the Baddeley model, the phonological loop comprises two components: a phonological store holding memory traces in acoustic form that fade in few seconds and an articulatory rehearsal process which refresh these memory traces using subvocal speech. As mentioned by Repovs & Baddeley (2006), information from other modalities enters the phonological store only through recoding into phonological form, a process preformed by articulatory rehearseal. In our working memory model, the student internal verbal information acquired through reading is assumed to be rapidly encoded in the episodic buffer and is manipulated in its semantic form instead of being recoded in the phonological store. Inasmuch, most of our learning laboratories concern mathematical subjects and several studies involving the Baddely working memory model suggest that the phonological loop participation in complex arithmetic is less important (De Rammelaere, Stuyven, & Vandierendonck, 1999; DeStefano & LeFevre, 2004; Seitz & Schumann-Hengsteler, 2000).

4.1. Overall view

The semantic and procedural knowledge of the learner are stored in the long-term memory. Concrete instances of this knowledge that must be addressed simultaneously during problem solving are activated in the working memory (WM). In our approach, the relation between the knowledge representation and the WM simulator is established via complex procedures. Each time the learner interacts with the laboratory interface, his/her actions are associated with primitive procedures and are interpreted as parts of complex procedures. While primitive procedures are seen as interface manipulations, execution of complex procedure represents the learner mental operations. Thus, the role of the WM simulator is to estimate the intrinsic cognitive load resulting from the execution the functions computing complex procedures result. The central executive computes the functions and the arguments needed to perform the computation are activated and accessed via the different working memory subcomponents.

Figure 3 illustrates the working memory simulator during the computation of a hexadecimal subtraction procedure with the following problem A-6.

Lon g Term Memory

next[A] = B next[B] = C next[10] = 11 next[11] = 12 A = 10 •—— 13-1.1 = 2 13-10 = 3

V

J

f

Sub

J

The function computed in the central executive is Sub and calculates the difference between the two numbers A and 6. This function only takes decimal arguments. The first argument (the hexadecimal number A has to be converted in decimal value using a previously automated function. Thus, the related knowledge instance (the decimal number 10) is directly accessed via long-term memory. The second argument (the decimal number 6) is visible through the laboratory interface and is accessed via the visuospatial sketchpad. A complete example of the working simulator is described in appendix A and the related mathematical details are exposed in appendix B.

4.2. Working memory load estimation

Within the field of cognitive psychology, working memory generally refers to "the system or mechanism underlying the maintenance of task-relevant information during performance of a cognitive task" (Miyake & Shah, 2002). Several models investigate working memory under the two functions of storage and processing of information (Anderson, Reder, & Lebiere, 1996; Baddeley, 1986; Cowan, 2005). Following most working memory models, our simulator includes these two processes in order to compute load estimations. As all theories assume a capacity limitation (Cowan, 2001; Miller, 1956), our simulator has a limited capacity on which rely storage and processing. These two components are implemented following a soft trade-off model of resource sharing.

Following the processing and storage resource sharing hypothesis, the intrinsic cognitive load estimation associated with the computation of a function within the working memory central executive is estimated using the next equation:

ICL - Pf x Sf (3)

l ^ L ri_ T [Func] [Func] v }

[Func]

where Pf (processing factor) and Sf (storage factor) are computed with equation 4 and equation 5. As proposed by Halford, G.S., Wilson W.H. & Philips, S. (1998), when a complex procedure engages many functions - as for intermediate calculations - the most demanding one is selected as the working memory load value. Taking the maximum value allows to simplify working memory load variation over time within one mental step.

4.3. Processing Factor

The processing complexity of a function is calculated as the number of interacting variables that must be represented in parallel to perform the task. In our computational model the central executive is responsible for computing functions and the variables are represented as instances of semantic knowledge. The processing factor for function execution is estimated using the next equation:

Args

-1

Pf = —

A (4)

[Func] 4 Func] ^

where args stands for the number of arguments needed to perform the function computation. Following the ACT-R. activation distribution principle (Anderson, 1993; Anderson, Reder & Lebiere, 1996), the central executive resources are equally distributed over the function arguments. Parameter A (computed with equation 2) stands for the function automation level and accounts for the positive effect of practice and learning on processing capacity (Halford, 1995; Halford, 1998). As suggested by Nikolic (2000), resulting from learning and practice, part of the limited processing capacity is freed for processing of additional information.

4.4. Storage Factor

The resource sharing between processing and storage is represented by the arguments accessibility of the different complex procedures functions. Each argument has an access probability depending on its functional location in working memory. The storage factor - which accounts for the effect of information maintenance on working memory load - is computed with the next equation:

Sf = — — - (5)

[Func] T T A c c e s s

A A [arg]

The access probability for each argument depends on its functional location (episodic buffer, long-term memory or visospatial sketchpad) and their related accessibility mathematical model. All access probability models (eq. 6 and 10) are bounded in a [0,1] range and therefore the storage factor minimum value is 1.

4.4.1. Episodic Buffer Access

As suggested by Baddeley (2003), knowledge instances representing intermediate results are created and stored in the episodic buffer. As indicated by equation 6, access probability in the episodic buffer depends on the two main theoretical accounts of information loss in working memory: interference and decay.

I [arg]

Episodic Buffer Access = (6)

K [arg] 1 + D r ,

1 T ^[arg]

where / (bounded in [0,1]) corresponds to the interference phenomenon (equation 7) and D (bounded in [0,oo]) corresponds to the decay phenomenon (equation 8).

The interference model developed by Oberauer & Kliegl (2006) states that items in working memory interfere with each other through interactions of their features. When different items share features, loss can occur while they compete for the same feature activation. Thus, interference results from the representational overlap between items. In our model, interference affects the probability of correctly recalling an item from the episodic buffer during a procedure argument access. The proportional feature loss effect is estimated with the following equation defined by Oberauer & Kliegl (2006):

I = (1-C/2)""

(7)

where n stands for the number of knowledge instances maintained in the episodic buffer and C to the average proportion of overwritten features between the item under retrieval and the n-1 others. The latter parameter is calculated regarding the different slots type of the semantic knowledge instances maintained in the episodic buffer. For example, if two items share 3

features among 6, then C equals 0.5. Equation 7 estimates the proportion of remaining feature after competing activations, therefore the interference phenomenon for an item under retrieval grows while the equation numerical value decrease.

The decay effect for memory retrieval within the episodic buffer follows researches on time-based resource sharing. According to Barrouillet, Bernardin & Camos (2004), decay is a function of the time during which a concurrent process totally captures attention and thus impedes refreshing. Memory traces to be recalled suffer from decay as soon as attention is switched away. The effect on maintenance is especially detrimental when concurrent tasks involve other memory retrievals. In order to take into account the effect of the concurrent functions execution time on argument retrieval probability, we modified the original decay model proposed by Barrouillet et al. (2004) and estimate decay with the next equation:

n m

D „ = n[\n<i-N)] (8)

i = 1

where U is the duration of the concurrent functions computed in the centre executive between the moment the argument is stored in the episodic buffer and the beginning of its retrieval process. As mentioned by Barrouillet, Bernardin, Portrat, Vergauwe & Camos (2007), the detrimental effect on maintenance of a concurrent process depends on its duration. Thus, the tt parameter

account for the trade-off effect between processing in the central executive and storage in the episodic buffer. The ay parameter represents the difficulty of each argument access involved in concurrent functions. As the ay parameter stands for the access probability of concurrent argument retrievals (computed with equation 6), 1-ay corresponds to their relative difficulty. The concurrent functions execution time (ti) is estimated with the following equation:

- j - J M if history > 0

[Func] I E otherwise ^ '

We estimate T, the time needed to mentally execute a function, using the mean time (M) needed by the learner to execute the same function explicitly via the laboratory interface . The E parameter is an estimate used when a learner has no explicit executions history for a function. In this case, the estimate represents the registered mean time for other learners regarding the same function.

4.4.2. Long-Term Memory Access

W h e n the result of intermediate computations is already k n o w n b y a learner, the associated

knowledge instances is directly retrieved from long-term memory (LTM) instead of being explicitly calculated in working memory. Following the Instance Theory of Automatization

1 Each function included in the different complex procedures defined in a learning laboratory has

(Logan, 1988; Logan, 2002), the stimulus perception triggers the retrieval process. In our approach, the stimulus is modeled with the function arguments. For example, "2+3=" acts as the retrieval cue to access "5 " in LTM. The next equation defines the arguments access probability resulting from long-term memory retrieval:

LTM A c c e s s = 0.9 A

x 11 Access (10)

[arg] [Func] 1 A [ a r g s ] v

In order to bind LTM retrieval cost with automation strength, variable A stands for the automation level of the function computing the argument and is calculated with equation 2. The access probability for each argument composing the stimulus retrieval cue is calculated depending on their functional location and related accessibility equation. As indicated by Logan (2004), storage and accesses occurring by means of long-term memory retrieval has low impact on processing.

4.4.3. Visual Access

When information is visually accessible, the related knowledge instances are stored in the visuospatial sketchpad. Within our current learning environments, visual information is always accessible. Thus, the visuospatial sketchpad maintenance mechanisms are not incorporated in the simulator. Instances of knowledge visually accessible are stored in the sketchpad visual cache as defined by Logie (1995). Consequently, the access probability of knowledge instances in the visuospatial sketchpad is always 1.0. As this paper focuses on the estimation of intrinsic cognitive load, visual access of function parameters has no trade-off effect with processing.

5. Cognitive Load Pattern Estimation

In order to optimise instructional settings, the tutoring system needs information about the cognitive load resulting from the execution of the different learning tasks proposed to a learner. More precisely, it needs to know how much germane cognitive load is available for the student during learning. In this sense the role of the cognitive patterns is to represent the cognitive load composition (intrinsic, extraneous and germane) undergone by the student at each resolution step when solving a given learning task.

As defined by equation 3, intrinsic cognitive load (ICL) - which results from the knowledge manipulation in working memory - is estimated with the working memory simulator. Depending on the student expertise level and its prior knowledge, the different arguments accessibility varies and provides personalised cognitive load estimations. For instance, during complex calculations, students having already automated a large number of simple calculation procedures access intermediate results via long-term memory instead of maintaining them in the episodic buffer. During learning, ICL measures for a same complex procedure will decrease in function of the progressive knowledge automation due to its utilisation.

Based on the assumption that the interface manipulations remain constant over all interface learning activity, the extraneous cognitive load (ECL) part of the cognitive patterns is estimated using a constant (10% of ICL).

Several researches indicate that knowledge encoding processes require more attentional resources than retrieval processes (Craik, Govoni, Benjamin & Anderson, 1996; Naveh-Benjamin, Craik, Perretta & Tonev, 2000; Naveh-Naveh-Benjamin, Guez & Sorek, 2007). In addition, Craik et al. (1996) demonstrated that memory performance tends to be more sensitive to disruptive tasks at encoding than retrieval. Recall may be autonomous and draw on cognitive resources while encoding is under subject's control and suffers from low attention. Following these researches, in our model the germane cognitive load (GCL) part of the cognitive patterns is assigned the remainder of the total cognitive capacity available as illustrated by the next equation:

GCL = L - (ICL + ECL) (11)

where L is a constant expressing the working memory capacity limit (see section 6). Craik et al. (1996) demonstrate that during procedure processing by the central executive, argument accesses always have priority over knowledge automation (encoding). Therefore, the computed GCL value accounts for the attentional factor during procedure automation - the GCL variable in equation 2 - and directly affects learning outcomes. As mentioned by Paas et al. (2003), an analytic estimation of the cognitive load composition allows a better distinction of the differentiated effect on learning of the three types of load during training activities.

6. Empirical Calibration

The constant L (equation 11) - which accounts for the working memory capacity limit - has been empirically calibrated during pre-utilisation phase.

Cognitive load researchers mostly rely on rating scales in order to evaluate the learner mental effort (Paas et al, 2003). These subjective techniques are based on the assumption that subjects are able to introspect on their cognitive processes and report the amount of mental effort expended (Gopher & Braune, 1984). A number of researches (cf. Paas, 1992; Paas & van MerrienBoer, 1994) have shown that people have no difficulty in assigning numerical values to the imposed mental load and that reliable measures can be obtained with unidimensional rating scales. We gathered subjective estimation of cognitive load using a 7-point rating scale (cf. Kalyuga, Chandler & Sweller, 1999; Pollock, Chandler & Sweller, 2002) where categories ranged from extremely low mental effort (1) to extremely high mental effort (7), accounting for overall cognitive load. A regression analysis was conducted between the analytical cognitive load estimations computed by the simulator and the subjective mental effort measures provided by the rating scales. The L constant is calculated by solving the resulting regression equation with the maximum subjective load measure (a value of 7 on the subjective rating scale). This alignment can be calculated during a calibration phase for each student using the tutoring system in order to individualise the model.

7. Conclusion

In this paper we presented a computational working memory simulator model for analytical cognitive load estimations within the context of intelligent tutoring systems. The simulator is

based on the Baddeley multicomponent model of working memory. During problem solving activities, the simulator is used to estimate the amount of mental effort invested by a learner to complete the different learning tasks. In order to be used by a tutoring system for pedagogical monitoring, the working memory simulator estimations are given a cognitive load theory interpretation. Cognitive load patterns are generated in order to describe the learner's mental effort composition regarding the three different components defined by the cognitive load theory (intrinsic, extraneous and germane load). The information provided by the cognitive patterns can be used to implement different pedagogical strategies relying on mental effort estimation (c.f. Courtemanche, Najjar & Mayers, 2008a; Courtemanche, Najjar & Mayers, 2008b). The model is empirically calibrated and a real class experiment is planned to validate the working memory simulator.

Acknowledgements

The authors want to thank (1) Yves Bouchard for his instructive comments on an early version of this paper and (2) Amir Abdessemed for his help on mathematical revision.

References

Anderson, J.R. (1993). Rules of the mind. Lawrence Erlbaum Eds.

Anderson, J.R., Reder, L. & Lebiere, C. (1996). Working memory: activation limitations on retrieval.

Cognitive Psychology, 30(3), 221-256.

Baddeley, A. (1986). Working Memory. Oxford, UK: Clarendon Press.

Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive

Sciences, 4(11), 417'-423.

Baddely, A. (2003). Working memory: looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829-839.

Barrouillet, P., Bernardin S. & Camos, V. (2004). Time constraints and resource sharing in adults' working memory span. Journal of Experimental Psychology: General, 133(1), 83-100.

Barrouillet, P., Bernardin S., Portrat, S., Vergauwe, E. & Camos, V (2007). Time and cognitive load in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(3), 570-585.

Clark, C , Nguyen, F. & Sweller, J. (2006). Efficiency in learning: evidence-based guidelines to manage

cognitive load. San Francisco, CA: Pfeiffer.

Courtemanche, F., Najjar, M. & Mayers, A. (2008a). Expertise Measure for Dynamic Task Selection within Intelligent Educational Systems. Proceedings of the 8th IEEE International Conference on

Advanced Learning Technologies (ICALT 2008), pp. 428-430. Santander, Spain.

Courtemanche, F., Najjar, M. & Mayers, A. (2008b). Cognitive Load Estimation for Optimizing Learning within Intelligent Tutoring Systems. Proceedings of the 9th International Conference on Intelligent

Tutoring Systems (ITS 2008), pp. 719-721. Montreal, Canada.

Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-185.

Craik, F. I. M., Govoni, R., Naveh-Benjamin, M. & Anderson, N. D. (1996). The effects of divided attention on encoding and retrieval processes in human memory. Journal of Experimental Psychology, 125(2), 159-180.

De Rammelaere, S., Stuyven, E. & Vandierendonck, A. (1999). The contribution of working memory resources in the verification of simple mental arithmetic sums. Psychological Research, 62(1), 72-77. DeStefano, D. & LeFevre, J. (2004). The role of working memory in mental arithmetic. European

Journal of Cognitive Psychology, 16(3), 353-386.

Gopher, D. & Braune, R. (1984). On the psychophysics of workload: Why bother with subjective measures? Human Factors, 26(5), 519-532.

Halford, G.S. (1995). Learning processes in cognitive development: A reassessment with some unexpected implications. Human Development, 38(6), 295-301.

Halford, G.S., Wilson W.H. & Philips, S. (1998). Processing capacity defined by relational complexity.

Behavioral And Brain Sciences, 21, 803-865.

Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351-371.

Logan, G.D. (1988). Toward an instance theory of automatization. Psychological Review, 95(4), 492-527. Logan, G.D. (2002). An instance theory of attention and memory. Psychological Review, 109(2),

376-400.

Logan, G.D. (2004). Working memory, task switching, and executive control in the task span procedure.

Journal of Experimental Psychology: General, 133(2), 218-236.

Logie, R.H. (1995). Visuo-spatial working memory. Hove, UK: Erlbaum.

Miyake, A., & Shah, P. (1999). Models of working memory: Mechanisms of active maintenance and

executive control. New York: Cambridge University Press.

Najjar, M. & Mayers, A. (2006). Retrieving of remembrances via a computational cognitive model of knowledge representation. In R. Sun (Eds.), Proceedings of the 28th Cognitive Science Society Annual

Meeting. Vancouver.

Najjar, M. & Mayers, A. (2007). AURELLIO: A cognitive computational knowledge representation theory. Int'l Journal of Cognitive Informatics and Natural Intelligence, 1(3), 17-35.

Naveh-Benjamin, M., Craik, F.I M., Perretta, J.G. & Tonev, S.T. (2000). The effects of divided attention on encoding and retrieval processes: The resiliency of retrieval processes. Quarterly Journal of

Experimental Psychology. 53A(3), 609-625.

Naveh-Benjamin, M., Guez, J. & Sorek, S. (2007). The effects of divided attention on encoding processes in memory: Mapping the locus of interference. Canadian Journal of Experimental Psychology, 61(1), 1-12.

Neely, J. H. (1989). Experimental dissociations and the episodic/semantic memory distinction.

Experimental Psychology: Human Learning and Memory, 6, 441-466.

Newell, A. (1990). Unified theories of cognition, Cambridge, MA: Harvard University Press.

Nikolic, D. (2000). A Dual processing theory of brain and mind: Where is the limited processing capacity coming from?. Proceedings of the first International Conference on Complex System.

Oberauer, K. & Kliegl, R. (2006). A formal model of capacity limits in working memory. Journal oj

Memory and Language, 55(4), 601-626.

Paas, F. & van MerrienBoer, J. J. G. (1994). Measurement of cognitive load in instructional research.

Perceptual and Motor Skills, 79(1), 419^130.

Paas, F., Tuovinen, J.E., Tabbers, H. & Van Gerven, P.W.M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63-71.

Paas, F., Renkl, A. & Sweller, J. (2004). Cognitive load theory: instructional implications of the interaction between information structures and cognitive architecture. Instructional Science, 31,1-8. Pollock, E., Chandler, P., & Sweller, J. (2002). Assimilating complex information. Learning and

Instruction, 12, 61-86.

Repovs, G. & Baddeley, A. (2006). The multi-component model of working memory: explorations in experimental cognitive psychology. Neuroscience, 139, 5-21.

Seitz, K. & Schumann-Hengsteler, R. (2000). Mental multiplication and working memory. European

Journal of Cognitive Psychology; 12(4),552-570.

Sweller, J., van MerrienBoer, J. J. G. & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296.

Tulving, E. (1983). Elements of episodic memory. New York: Oxford University Press.

van Merrienboer, J.J.G. & Sweller, J. (2005). Cognitive load theory and complex learning: recent developments and future directions. Educational Psychology Review, 17(2), 147-177.

Appendix A

This appendix describes an example of the working memory simulator using a hexadecimal subtraction complex procedure (subtracting C from E). As illustrated in figure 4, the learner computes the result algorithmically in a single mental process composed of three functions.

Working Memory

Long Term Memory

A - 1 0 nextJA) - 8 mxtlB] - C rt«xt[IO)~ 11 n e x t ( l l ) * 1 2 1 4 - 1 4 = 0 14-13 = 1 Central executive S + SCWtV «.>.2 + (conv 1* t ** 5 ~j~

LH

I

It

Figure 4 - Illustrative example

At time t=0, the current function - which is converting E to its decimal value - is not automated. The function is computed in the central executive and the result is stored in the episodic buffer. The single argument of the function is visible in the laboratory interface and is accessed via the visuospatial sketchpad. The associated intrinsic cognitive load (ICL) estimation is 0.5112 (see appendix A-part 1 for details). The same settings account for the function computed at t=2 and the associated ICL estimation is 0.5096 (see appendix A-part 2 for details). At time t=4, the function computed in the central executive is subf 14,12]. This function is already automated and the result is directly retrieved from long-term memory. The two arguments - accounting for the retrieval cue - are the intermediate results previously calculated and are accessible via the episodic buffer (see figure 4). The associated ICL is 7.4738. Here, most of the ICL accounts for arguments' maintenance in the episodic buffer (see appendix A-part 3 for details). In this case, the third step of the complex procedure is the most demanding one. Thus, for this example, the estimated ICL is taken from the function sub[14,12] and has a value of 7.4738.

Appendix B Parti (eq. 2) A N '[func] ( N + 1) 0.958 ' 0.958+1 ' 0.489 -0.5 N = Z (GCL x i i = n = <•> (0.45x1.0) + (0.37x0.717) + (0.42x0.577) = 0.958

(*2 The GCL variable have been previously calculated at each encoding using equation 11

(eq. 4) P f Args

| Part 3 I

(eq. 8) D - f t IT, fi(1-a „) ] 112J i - i i = i ' = 0 (eq- 7) I[ 1 2 ]= ( 1 - C / 2 ) " "1 = (1 - 0 . 5 / 2 )2"1 (••) = 0.75 x A [func] 4 1 -1 = — x 0.489 4 = 0.5112 '[func] (eq. 5) S f [func] f j Access [arg] Access [El (eq. 3) ICL [func] 1 1 Pf x Sf [func] [func] 0.5112 1 1 1 0.5112 Part 2 (eq. 2) A N (eq. 4) Pf [func] (N + 1) . 0.963 ' 0.963+1 = 0.4906 _ Args N = Z (GCLiXr05; i = n = (•) (0.45x1.0) + (0.37x0.717) + (0.43x0.577) = 0.963 [func] 4 1 x A 4 = 0.5096 x 0.4906 -1 [func] • 1 (eq- 5) Sf _{r = n}

[func] X1 Access [arg] Access, '[C] (eq. 3) ICL [func] Pf x Sf [func] [func] 0.5096 1 = — I " " I " = 0.5096

(eq-8)Dri4,= ,ft ITft/l-aij) = [ ( 2.01 ) ] = 2.01 (eq. 7) I = (1 - C/2) " "1 [14] ( 1 - 0 . 5 / 2 ) 2"1( « ) (eq. 6) E B A c c e s s = '[12] [12] T T F _ 0.75 0.75 '[12] 1 + 0 = 0.75 (eq. 6) E B A c c e s s _ '[12] [14] '[12] p., C = 0.5 : both instances share one feature

among two, which is the type (integer)

1 +D| _ 0.75 " 1 + 2 . 0 1 = 0.249 (eq. 2) A, N '[funcf (N + 1) . 3.90 ' 3.90+1 •• 0.7959 N = Z ( G C L i X f0 5) i = n = (*) (0.92x1)+ (0.88x0.717) + (0.90x0.577) + (0.83x0.5) + (0.87x0.447) + (0.91x0.408) • (0.91x0.378) + (0.88x0.354) = 3.90

(.) The GCL variable have been previously calculated at each encoding using equation 11

(eq. 10) L T M A c c e s s = 0 . 9 A „ . x n A c c e s s , , [arg] lf u n cl larSl = 0.9 x 0.7959 X (0.75 X 0.249) = 0.1338 (eq. 3) ICL = Sf , [func] [fun<=] 1 0.1338 7.4738

Chapitre 2

La boucle interne

L'objectif principal de la boucle interne d'un STI concerne Panalyse des etapes de resolution

d'un problems. Pour remplir ce role, celle-ci est responsable de fournir a l'etudiant differents

services au cours de sa periode d'apprentissage. De maniere generate, on retrouve les cinq

services suivants dans la boucle interne de la majorite des systemes tutoriels [14].

Retroaction minimale: la retroaction minimale a pour but d'indiquer a l'etudiant si sa

derniere etape de resolution est correcte ou incorrecte. La retroaction peut etre immediate,

differee ou fournie sur demande de l'etudiant.

Indice : lorsque l'apprenant est bloque ou qu'il en fait la demande, le systeme peut lui fournir

un indice sur la prochaine etape a effectuer. Generalement, les indices sont donnes lors de

l'occurrence de certaines situations preetablies (blocage, erreurs repetees, etc.).

Retroaction specifique: lorsque le systeme est en mesure de determiner les inferences

erronees qui ont mene l'apprenant a une etape incorrecte, il donne des instructions

specifiques concernant ces inferences.

Evaluation des connaissances : le systeme evalue le niveau de maitrise des connaissances

en fonction des strategies adoptees par la boucle externe (cf. chapitre 3). Le type

d'information requise pour le choix des problemes determine la granularite de 1'evaluation.

Revision de la solution : lorsque l'etudiant a effectue plusieurs etapes erronees, le role

2.1 Apprentissage supervise

L'apprentissage supervise est une strategic pedagogique basee sur l'hypothese qu'on observe

un apprentissage accru lorsque l'apprenant est guide durant sa resolution de probleme [6].

L'explication vient du fait qu'en suppleant a l'effort mental de l'etudiant, en lui donnant des

indices sur les prochaines etapes, des ressources cognitives contribuant davantage a

l'apprentissage sont liberees.

L'article presente dans ce chapitre explique la maniere dont l'estimation de la charge

cognitive fournie par le modele de l'apprenant est utilisee au profit de la boucle interne.

L'approche repose sur la mise en place d'une strategic d'apprentissage supervise, basee sur la

theorie de la charge cognitive, ou l'apprenant est guide par des indices sur la prochaine etape

de resolution a effectuer.

L'article a ete publie dans une version courte {short paper) a la 9

conference ITS

(International Conference on Intelligent Tutoring Systems), qui a eu lieu du 23 au 27 juin

2008 a Montreal. Cette derniere version peut etre consultee aux pages 719-721 des actes de la

conference. La version entiere, telle que soumise initialement, est presentee dans ce memoire.

Cognitive Load Estimation for Optimizing Learning

within Intelligent Tutoring Systems

Fran9ois Courtemanche1, Mehdi Najjar2 and Andre Mayers1

'ASTUS Research Group, Department of Computer Science, University of Sherbrooke interdisciplinary Research Center on Emerging Technologies, University of Montreal

{f.courtemanche, mehdi.najjar, andre.mayers}@usherbrooke.ca

Abstract. This paper presents a guided learning strategy model for dynamic pedagogical tailoring within intelligent tutoring systems (ITS). The proposed model is based on the integration of the cognitive load theory framework within an ITS architecture. Our approach takes into account the cognitive limitations and the expertise level of the student in order to offer personalised learning via instructional guidance. The article also discusses distinctions between our approach and related works.

1 Introduction

Most intelligent tutoring systems (ITS) use pedagogical objectives or performance measures in order to adapt tutoring strategies [2,9,27]. Several researches in educational psychology suggest that highly effective instruction can only be attained by taking into account the learner cognitive constraints [12,25]. This paper introduces a novel approach for guided learning within ITS which aims to dynamically adapt instruction in order to respect the student cognitive limitations. The approach is based on the cognitive load theory (CLT) framework and uses a working memory simulator. The remainder of the paper is organised as follows. Section 2 exposes the CLT framework. Section 3 expounds the main architecture of our ITS and describes the integration of the CLT concepts. Following a presentation of the virtual learning environment developed to test the approach, we explain in section 5 how the CLT framework and the working memory simulator are used in order to implement guided learning in our tutoring system. After reporting on early-pilot tests in section 6, concluding remarks are given in section 7.

2 The Cognitive Load Theory

The cognitive load theory (CLT) is a framework representing characteristics of the mental effort that results from the performance of complex cognitive tasks during learning [6,22,24]. CLT is based on the interaction between the knowledge structures

and the human cognitive architecture. The CLT framework aims to identify optimal methods of instruction regarding human cognitive limitations.

CLT considers the two types of memory which are widely accepted in the cognitive psychology community. The working memory (WM) is a dynamic memory serving as a temporary storage for cognitive tasks such as reasoning, understanding and learning. This is where is hold all information mentally manipulated by a learner. WM can hold for processing about 4 ± 2 elements simultaneously [7]. The long-term memory (LTM) is defined as a static structure of infinite capacity, which keeps the knowledge and skills accumulated over the years. In LTM, information is organised in the form of schemas. The latter are used in order to bind together elements of knowledge handled to achieve common goals.

The capacity limit of working memory can be expressed as the number of schemas treated simultaneously. Size and complexity of these schemas are not relevant regarding capacity limit. The creation of schemas reduces the burden on working memory by combining several knowledge items into a single one. Whereas a domain expert has a significant amount of specialised schemas allowing him/her to solve complex problems by limiting his working memory load, a novice - due to the absence of appropriate schemas - must engage various resolution strategies (e.g. mean-end analysis) [22]. This often results in memory overload and subsequent failing.

Working memory load is defined by three components: intrinsic cognitive load (ICL), extraneous cognitive load (ECL) and germane cognitive load (GCL). ICL represents the interaction between the knowledge to be learned and the expertise level of the learner. This load is imposed by the number of elements to be addressed simultaneously in working memory. This number depends in turn on the knowledge aggregation degree (expertise) in LTM. For a same task, ICL varies according with the knowledge automation level of the learner. Well structured knowledge implies less knowledge units to be handled in working memory and thus, lower ICL. ECL represents all form of load which is not directly devoted to knowledge manipulation (e.g. interface interactions, attention splitting). This load is not effective for learning and can be reduced by a better instructional design. GCL represents the load resulting from learning processes (automation, schema acquisition).

These three types of cognitive load are additive. Their sum may not exceed the working memory capacity without causing a failure of the ongoing task. However, they can be balanced to maximize learning. For example, taking into account that, for a given task, intrinsic cognitive load is fixed, extraneous cognitive load can be reduced. ECL reduction can be achieved by better design of interface in order, for example, to avoid switches of attention. This leaves more resources for germane cognitive load and thus foster schemas acquisition [6]. Other techniques such as the

isolated-elements strategy and hierarchical learning (see section 5.3) intend to reduce

3 ITS and the CLT Framework

One of the key issues of the cognitive load theory (CLT) is to formalise the interaction between the knowledge manipulated by a learner and the cognitive constraints related to learning. These two aspects can be modeled and adjusted in an ITS to optimize learning. This section shows how our ITS allows embedding CLT using a simulated working memory and a flexible knowledge representation.

3.1 The Architecture

Similarly to most intelligent tutoring systems [28], our ITS architecture includes four main modules. Figure 1 shows an overview of the architecture.

Laboratory simulation kernel

Laboratory

visual interface Learner

Figure 1 - The architecture

The expert agent is able to dynamically solve any problem proposed to a student. The main role of the expert agent is to provide the best solution to a given problem.

The interface agent represents the link between the actions carried out in the virtual learning environment by the learner and the other modules of the architecture. The main function of the interface agent is to connect the physical interactions made in the virtual environment with the domain knowledge.

The learner agent encodes the learner knowledge and his behaviour (actions) as episodes (section 3.2). In addition, our learner agent extends these standard capabilities by modeling the characteristics of the cognitive effort (cognitive load) of the learner during an episode. Our learner agent is able to achieve this task thanks to a simulated working memory (section 3.3).

The pedagogical agent provides educational remediation in response to the learner behaviour and implements the usual pedagogical features of feedback and hints on the next step [27]. The originality of our pedagogical agent is its ability to take into account dynamically, via the learner model, the cognitive effort imposed by

the various resolution steps of a learning task. Most ITSs pedagogical agents rely on the information provided by the expert module to choose next step hints [cf. 2,9,26]. The hints for the next step on the solution path provided by EA are selected regardless of the learner knowledge evolution. In our approach, the next step hints provided on demand and for guidance purposes is dynamically determined considering the present learner expertise level and mental load.

The laboratory resources contain the learning tasks and the domain knowledge of a virtual laboratory. The domain knowledge is expressed in terms of semantic [18] and procedural knowledge [1], Information concerning the learner expertise level is stored in the learner resources.

Our ITS behaves like a conventional cognitive tutor [2] by 1) determining all valid actions at each problem step, 2) distinguishing between correct and erroneous actions, and 3) providing the next best step. However, thanks to the learner model and the expert agent, our ITS can go further in its cognitive support to the learner.

3.2 The Knowledge Representation

Our knowledge representation model is implemented as a dynamic graph including three types of memory structure and knowledge, generally accepted in the cognitive psychology literature [16]: semantic knowledge, procedural knowledge and episodic knowledge.

Semantic knowledge represents a concept taken in broad sense where it can be any category of objects [17].

Procedural knowledge is used to satisfy needs without using the attention system [1]. In our approach, procedures are subdivided in two main categories: primitive procedures and complex procedures. The former are seen as atomic actions on semantic knowledge and are reified in the system as interactions of the learner via the laboratory interface. The execution of the latter is done by sequences of actions, which satisfy scripts of goals. The learner agent, which represents the learner knowledge and his behaviour, estimates the automation level of complex procedures to assess the impact on the working memory (section 3.3).

Episodic memory retains details about our experiences and preserves temporal relationships [23]. In our approach, the episodes are based on instantiation of goals (which are seen as generic statements retrieved from the semantic memory). The episodic knowledge is organized according to goals and the procedures needed to achieve these goals. Each goal realisation is explicitly encoded in an episode.

3.3 T h e W o r k i n g M e m o r y M o d e l

The working memory (WM) simulated by the learner agent is inspired by the Baddeley model [4,20]. Figure 3 illustrates the four modules defined by the model which interact to perform a variety of complex cognitive tasks. The central executive coordinates the simultaneous task executions and oversees the other modules. This module is also responsible of the knowledge retrievals from the long-term memory.