Exploring the Impact of Extroversion on the Selection of Learning Materials

(1)

Exploring the Impact of Extroversion on the Selection of Learning Materials

Manal Alhathli

University of Aberdeen Aberdeen, UK

r01maea@abdn.ac.uk

Judith Masthoff

j.masthoff@abdn.ac.uk

Advaith Siddharthan

advaith@abdn.ac.uk

ABSTRACT

The Internet provides access to many learning materials that could complement class room teaching. An educational recommender system can aid learners to find learning materials most suitable to them. The best learning materials will de- pend on learner characteristics. This paper investigates the influence of learner personality. In particular, it describes a study in the language learning domain that explores the relation between learners’ extroversion and the extent to which learning materials are perceived to be enjoyable and to increase their confidence and skills. We found positive correlations between extroversion and these criteria for social and active learning materials.

Keywords

Personality, learning material, educational recommender, personalization

1. INTRODUCTION

E-learning has been gaining in importance, as also seen by the plethora of massive open online courses. In addition to established courses, there is a vast quantity of learning materials available on-line, such as YouTube videos. For learners, it can be quite difficult to find the materials best for them.

Educational recommender systems may help to solve this problem. To be effective, such systems need to personalize their recommendations to learner characteristics, to ensure recommended materials suit individual users with different needs and requirements.

Our research aims to discover if a recommender system that incorporates learners’ psychological traits in it deci- sion making would prove more effective than current applications. Therefore, this paper’s study aims to find evidence of whether a learner’s personality, in particular their degree of extroversion, should have an impact on the selection of learning materials. This work will feed into future work on constructing adaptive educational recommendation mecha- nisms specifically tailored to learners’ personalities.

2. RELATED WORK

2.1 Educational recommender systems

Educational recommender systems aim to improve the learning process by recommending appropriate courses, top- ics, peers, or learning materials (see [14] for an overview).

We are particularly interested in recommendation of learning materials. For example, [13] have developed an on-line personalized English learning recommender system that provides reading materials for English Second Language learners. The educational recommender of [2] provides useful information and pages from the Internet for observed gaps in learners’ knowledge. The ISIS system recommends learning activities (as part of a Psychology course), providing navigation support in self-organized learning networks [8].

The educational recommender of [28] recommends learning documents from a set provided by the teacher and also dis- cusses extending this to recommend text books. The educational recommender of [11] makes suggestions about similar materials based on learners’ ratings to enhance e-learning performance. The CoFind system [9] recommends learning materials based on folksonomies. The Altered Vista system [21] and QSIA system [20] recommend learning resources to members of a learning community.

2.2 Personality in learning

Several publications have documented five dimensions (i.e.

extroversion, openness, conscientiousness, agreeableness and neuroticism) as underpinning the basic characteristics of personality, providing practical methods to appraise individuals [15]. Several studies have shown that there is a strong rela- tionship between personality and academic performance [23]

For example, Openness to Experience was associated positively with academic performance [1]. Extroversion shows a positive correlation with participation in academic activities such as seminars [10] and low positive correlation with final exam grades [3]. Conscientiousness shows a strong positive correlation with final exam grades [3]. Furthermore, creativity in learning has been positively correlated with extroversion [22]. Another positive correlation can be found between four personality traits (extroversion, agreeableness, conscientiousness, openness) and a student’s motivation to attend college [4]. Given the clear evidence that personality influence learning, we would like to investigate how educational recommender systems can take personality into account.

There has a been strong tradition in e-learning to personalize interactive instruction systems to learners, leading to for example Intelligent Tutoring Systems. However, most

(2)

of these systems adapt to other learner characteristics, for example to performance, affective state, and learning styles (which differ from personality). Research on adapting e- learning systems to personality has been more limited. Den- nis et al [7] investigated adapting learner performance feedback and emotional support to the Big5 personality traits.

Okpo et al [19, 18] investigated adapting exercise difficulty to learner self-esteem. Robison et al [24] investigated the role of personality when giving different feedback types to learners. Del Soldato and Du Boulay [5] describe a motivational planner that can adapt its tactics to learner confidence. Nunes [17] describes a recommender that recommends compatible learners to work with based on Big5 personality traits. To the best of our knowledge, there is no previous work on recommending learning materials based on personality.

3. STUDY: IMPACT OF EXTROVERSION 3.1 Study Design

The study investigates which types of learning material are best for those with different personalities, in particular extroversion. There were three parts to the study. The first section gathered basic demographic information from participants and contained a short personality test. The second part asked participants to rate the learning materials for “John”, a fictional foreign language learner, who was described as having a similar personality to the participant.

The final part asked participants to pick the best learning material.

3.1.1 Participants

The study was administered as a questionnaire on Me- chanical Turk [16]. We included a Cloze Test [29] for En- glish fluency to ensure that workers possessed enough liter- acy skills to understand the language based nature of the task. Participants had to have an acceptance rate of 90%, be based in the US and pass the fluency test in order to be eligible for the study. There were 50 participants (14 fe- male, 35 male, 1 non-disclosed; 9 aged 18-25, 28 aged 26-40, 13 aged 41-65).

3.1.2 Materials

Foreign language learning was chosen as the domain, as many learning materials for it exist and it lends itself easily for different types of learning materials. Food ordering in a restuarant was chosen as the topic, as this is very popular in language courses. We used 7 learning materials, which a short textual description provided to participants (see Table 1). Learning materials were intended to be either passive or active, and individual or social. This was validated during the study by participants rating the extent to which the material involved John in active participation (active), and the extent to which it involved John in social interaction (social).

3.1.3 Variables

The independent variables are: the personality of ‘John’

(which matched the personality of the participant) focusing on extroversion, whether the learning material was active or passive, and whether the learning material was individual or social.

Table 1: Learning materials ID Learning Materials

1 In this learning material, John will participate in an on-line spoken dialogue with a fellow learner about ordering food. John will play the role of the customer and the fellow learner the role of the waiter.

2 In this learning material, John will participate in an on-line spoken dialogue with a native speaker about ordering food. John will play the role of the customer and the native speaker the role of the waiter.

3 In this learning material, John will participate in an on-line spoken dialogue with a virtual agent (computer) about ordering food. John will play the role of the customer and the virtual agent (computer) the role of the waiter.

4 In this learning material, John will view a video about two native speakers having a dialogue in a restaurant. Next, the dialogue will be translated into John’s own language.

5 In this learning material, John will view a video about two other learners having a dialogue about ordering food in a restaurant.

6 In this learning material, John will view a video showing two other learners having a dialogue about ordering food in a restaurant. John can provide spoken feedback to the learners on their performance.

7 In this learning material, John will practice the food ordering vocabulary using multiple choice ex- ercises.

The dependent variables are: the extent to which participants felt the learning material is enjoyable for John, in- creases John’s confidence in the language, improves John’s language skills, and the most preferred (‘best’) learning material to use for John. We will abbreviate the three ratings to enjoyable, confidence and skills below. Each of these ratings was given on a 5 point Likert-scale, from “not at all” to

“a lot”.

3.1.4 Procedure

Participants first completed the English fluency test. If they passed, they provided their demographics and took a short personality test for the Five-Factor Model (FFM) [12], usingPersonality Sliders, a newly developed personality test [26]. For each trait from the FFM, participants were shown two stories (developed by [6]), one depicting a person that was low for that trait and the other depicting someone who was high. Participants used a slider to indicate which person they were most like, resulting in a value for each trait between 18 and 162. These are validated as accurately measuring the FFM [27].

On the next screen, participants were introduced to “John”, who has a similar personality to them. They were told that John is learning a foreign language and has just attended a class on ordering food in a restaurant. Next, they rated each learning material in turn in random order using the 5 scales (enjoyable, confidence, skills, active, social). Finally, they selected the learning material which was best for John.

(3)

3.2 Results

3.2.1 Types of learning materials

We first tested whether the learning materials did indeed match the individual versus social, and passive versus active distinctions we anticipated. Results are shown in Table 2.

We conducted one sample t-tests to investigate whether the mean was significantly different from the mid-point of the scale (i.e. 3, see table for significance values). Based on this, four learning materials were found to be active (learning materials 1, 2, 3, 6), two passive (4 and 5), and one neither active nor passive (7). Three learning materials were found to be social (1, 2, 6), three individual (4, 5, 7), and one neither (3).

Table 2: Mean (stdev) of active and social ratings Learning material Active Social

1 4.42 (.73)*** 4.26 (.83)***

2 4.28 (.88)*** 4.22 (.91)***

3 4.06 (.96)*** 3.04 (1.21)

4 2.20 (1.09)*** 1.96 (1.03)***

5 1.90 (1.00)*** 1.80 (.99)***

6 3.52 (1.02)** 3.44 (.99)**

7 2.92 (1.23) 1.98 (1.17)***

**p<.01, ***p<.001

3.2.2 Extroversion and learning material ratings

First, we investigated the Pearson correlation between participants’ level of extroversion and their ratings forpas- siveandactive learning materials respectively. The results are shown in Tables 3 and 4. For passive learning materials, we found no significant correlations. For active learning materials, we found significant and positive correlations of extroversion with (1) the enjoyability of learning materials, (2) whether the learning material increased learner’s confidence, and (3) whether it increased learner’s language skills.

These results may be explained by the fact that most active learning materials were social, whilst the passive ones were all individual.

Table 3: Correlations for passive learning materials Enjoyable Confidence Language-skills

Extroversion .165 .068 .053

Table 4: Correlations for active learning materials Enjoyable Confidence Language-skills Extroversion .275** .179* .160*

*p<.05, **p<.01

Secondly, we investigated the correlations between participants’ level of extroversion and their ratings forindividual and social learning materials respectively. The results are shown in Tables 5 and 6. For individual learning materials, we found no significant correlations. For social learning materials, we found significant and positive correlations of extroversion with (1) the enjoyability of learning materials, (2) whether the learning material increased learner’s confidence, and (3) whether it increased learner’s language skills.

These results can be explained by extroverts preferring social learning materials.

Table 5: Correlations for individual materials Enjoyable Confidence Language-skills

Extroversion .158 .011 .017

Table 6: Correlations for social learning materials Enjoyable Confidence Language-skills Extroversion .315** .206* .195*

*p<.05, **p<.01

Next, we split the participants into two groups, based on their score for extroversion: an extrovert group with an extroversion score above the mid point of the scale, and an introvert group with a score below the mid point of the scale. There were 17 extroverts and 33 introverts. Rat- ings per group for the different types of learning materials are provided in Tables 7-10.

Table 7: Mean (stdev) for social materials Enjoyable Confidence Skills Extroverts 3.49 (.67) 3.86 (.83) 3.98 (.93) Introverts 2.87 (1.12) 3.49 (1.02) 3.69 (.97)

Table 8: Mean (stdev) for individual materials Enjoyable Confidence Skills Extroverts 3.04 (1.04) 3.04 (1.08) 3.16 (1.12) Introverts 2.81 (1.24) 3.13 (1.20) 3.24 (1.12)

Table 9: Mean (stdev) for passive materials Enjoyable Confidence Skills Extroverts 3.06 (.98) 3.06 (1.13) 3.03 (1.17) Introverts 2.80 (1.27) 2.98 (1.25) 3.06 (1.15)

Table 10: Mean (stdev) for active materials Enjoyable Confidence Skills Extroverts 3.53 (.74) 3.84 (.86) 3.97 (.91) Introverts 3.00 (1.11) 3.58 (1.02) 3.77 (.94) Considering enjoyment, the extrovert group rated both active, passive, social and individual learning materials as more enjoyable than the introvert group, however the differences between extroverts and introverts are only significant for social and active materials (p<.05), in line with the correlation results.

Considering increasing the learner’s confidence, the extrovert group rated the social learning materials a lot higher than the individual ones. The introvert group also rated social materials more highly, but the difference was a lot smaller than for the extroverts. Interestingly, extroverts rated social materials higher than introverts, but rated individual materials lower, but this was not statistically significant. Looking at the difference between active and passive

(4)

learning materials, both extroverts and introverts rated the active materials a lot higher than the passive ones, and in both cases the extroverts had slightly higher ratings than the introverts (though not significant).

Considering increasing the learner’s language skills, the extrovert group rated the social learning materials a lot higher than the individual learning materials. The introvert group also rated social materials more highly, but the difference was a lot smaller than for the extroverts. Interestingly, extroverts rated social materials higher than introverts, but rated individual materials lower. Looking at the difference between active and passive learning materials, both extroverts and introverts rated the active materials a lot higher than the passive ones. The difference between extroverts and introverts for passive materials is negligible, whilst for active materials extroverts rated slightly (but not significantly) higher.

3.2.3 Extroversion and learning material selection

Figure 1 shows the selection of the best learning material among the two groups of introverts and extroverts. It can be clearly seen that the majority of the two groups find that social and active materials are the best to recommend to John (though interestingly, learning material 6 which is both social and active is completely absent). The second most selected learning materials in introverts were passive and individual. Interestingly learning materials 5 and 7 (both individual and neither active) were not selected at all in the extroverts’ group, whilst they were selected in the introverts group. There also seems to be an interesting difference for learning material 3: this was the material where learners had a dialogue with a virtual agent (instead of with a human as in learning materials 1 and 2). This seems more popular with introverts than extraverts. More statistical analysis can be done here.

4. CONCLUSIONS AND FUTURE WORK

In this paper we presented a first study on how a computer can recommend learning materials to users and investigated in particular the effect of learners’ extroversion on their appreciation of learning materials on three criteria. We presented an initial analysis of the data, which showed that extroversion was weakly but positively correlated for active and social learning materials with learners’ rating of how much they thought a learner with a personality similar to their own would (1) enjoy the learning material, (2) increase their confidence through the learning material, (3) increase their skills through the learning material. Further statistical analysis showed significant differences between introverts and extroverts for enjoyment. Future work will extend the analysis of the data and will use the results of this study and follow-on studies to inform the design of adaptive recommendation algorithms.

This paper presented an indirect study, we did not mea- sure actual enjoyment, actual increase in confidence and actual increase in skills, but perceptions of those. In a sense, therefore the study looked at learner preferences. Clearly there is more to an effective educational recommender system than learner preferences (as noted by [25]). We plan to run follow-on studies in a real learning environment to obtain more direct measures, also on learning gain. In addition, we plan to interview teachers to obtain their input.

This paper only investigated extroversion. We plan to

Extroversion

Extrovert Introvert

Count

12

10

8

6

4

2

0

7 5 4 3 2 1 Learning material

Page 1

Figure 1: Selection of the best learning material.

Note that counts in extroversion and introversion groups differ. The colour indicates the social versus individual aspect of learning materials: blue=social, individual=yellow, neither=green. The pattern indicates the active versus passive aspect of the learning materials: no pattern=active, stripes=passive, dots=neither.

conduct other studies to investigate other personality traits as well as their interaction with other learner characteristics (e.g. learners’ goals and interests, knowledge and performance, learning styles, age). In this study, we recruited participants via Amazon Mechanical Turk, meaning they all came from the US. It is possible that the preference for social and active learning materials is related to participants cultural backgrounds, and this can be investigated in future studies.

The study used the domain of learning a foreign language.

This may have had an impact, as skills’ and confidence development in language learning may benefit more from social interactions. We plan to repeat the study in another learning domain. In this study, we only distinguished between active/passive and social/individual learning materials, as we assumed that those were the most important for the extroversion personality trait. The learning topic was the same for all learning materials, and difficulty of the learning materials was not considered. Other learning material characteristics (and their interactions) still need to be investigated.

For example, it is possible that for openness to experience the novelty or diversity of learning material may be relevant (c.f. [30] for adaptations of recommender systems’ diversity to openness of experience).

Finally, we did not investigate different ways to present recommendations to learners. It is possible to use for example Top-N recommendations, presenting multiple ranked materials for learners to choose from, or to indicated the expected suitability of a learning material for instance by stars. An overview of this issue for recommenders in general can be found in [31], and more work on presentations and explanations of educational recommendations is required.

(5)

5. ACKNOWLEDGMENTS

We thank the Ministry of Higher Education in Saudi Ara- bia for sponsoring the PhD of the first author.

6. REFERENCES

[1] G. Blickle. Personality traits, learning stratigies, and performance.European Journal of Personality, 10(5):337–352, 1996.

[2] C.-N. Bodea, M.-I. Dascalu, and A. Lipai. Clustering of the web search results in educational recommender systems.Educational Recommender Systems and Technologies: Practices and Challenges: Practices and Challenges, page 154, 2011.

[3] T. Chamorro-Premuzic and A. Furnham. Personality predicts academic performance: Evidence from two longitudinal university samples.Journal of Research in Personality, 37(4):319–338, 2003.

[4] M. Clark and C. A. Schroth. Examining relationships between academic motivation and personality among college students.Learning and individual differences, 20(1):19–24, 2010.

[5] T. del Solato and B. Du Boulay. Implementation of motivational tactics in tutoring systems.Journal of Interactive Learning Research, 6(4):337, 1995.

[6] M. Dennis, J. Masthoff, and C. Mellish. The quest for validated personality trait stories. InProceedings of IUI 2012, pages 273–276, New York, USA, 2012. ACM.

[7] M. Dennis, J. Masthoff, and C. Mellish. Adapting progress feedback and emotional support to learner personality.International Journal of Artificial Intelligence in Education, pages 1–55, 2015.

[8] H. Drachsler, H. G. Hummel, B. Van den Berg, J. Eshuis, W. Waterink, R. Nadolski, A. J. Berlanga, N. Boers, and R. Koper. Effects of the isis

recommender system for navigation support in self-organised learning networks.Educational Technology & Society, 12(3):115–126, 2009.

[9] J. Dron, C. Boyne, R. Mitchell, and P. Siviter. Cofind:

steps towards a self-organising learning environment.

InWebNet, pages 146–151, 2000.

[10] A. Furnham and S. Medhurst. Personality correlates of academic seminar behaviour: A study of four instruments.Personality and Individual Differences, 19(2):197 – 208, 1995.

[11] K. I. Ghauth and N. A. Abdullah. Learning materials recommendation using good learners’ ratings and content-based filtering.Educational Technology Research and Development, 58(6):711–727, 2010.

[12] L. Goldberg. The structure of phenotypic personality traits.American Psychologist, 48:26–34, 1993.

[13] M.-H. Hsu. A personalized english learning

recommender system for esl students.Expert Systems with Applications, 34(1):683–688, 2008.

[14] N. Manouselis, H. Drachsler, R. Vuorikari,

H. Hummel, and R. Koper. Recommender systems in technology enhanced learning. InRecommender systems handbook, pages 387–415. Springer, 2011.

[15] R. McCrae. The five-factor model: Issues and applications [special issue].Journal of Personality, 60(2):175–215, 1992.

[16] MT. Amazon mechanical turk.

http://www.mturk.com, 2012.

[17] M. A. S. N. Nunes.Recommender systems based on personality traits. PhD thesis, Universit´e Montpellier II-Sciences et Techniques du Languedoc, 2008.

[18] J. Okpo. Adaptive exercise selection for an intelligent tutoring system.In proceedings of the 24th

International Conference on User Modeling,

Adaptation and Personalization (UMAP’16), In press.

[19] J. Okpo, M. Dennis, K. A. Smith, J. Masthoff, and N. Beacham. How can skin check reminders be personalised to patient conscientiousness? Proceedings of the 13th International Conference on Intelligent Tutoring Systems, In Press.

[20] S. Rafaeli, M. Barak, Y. Dan-Gur, and E. Toch.

Qsia–a web-based environment for learning, assessing and knowledge sharing in communities.Computers &

Education, 43(3):273–289, 2004.

[21] M. M. Recker and A. Walker. Supporting”

word-of-mouth” social networks through collaborative information filtering.Journal of Interactive Learning Research, 14(1):79, 2003.

[22] A. G. Richardson. Personality correlates of creativity among a sample of jamaican adolescents.Personality and Individual Differences, 6(6):771–774, 1985.

[23] H. Rindermann and A. C. Neubauer. The influence of personality on three aspects of cognitive performance:

Processing speed, intelligence and school performance.

Personality and Individual Differences, 30(5):829–842, 2001.

[24] J. Robison, S. McQuiggan, and J. Lester. Developing empirically based student personality profiles for affective feedback models. InIntelligent Tutoring Systems, pages 285–295. Springer, 2010.

[25] O. C. Santos and J. G. Boticario. Modeling

recommendations for the educational domain.Procedia Computer Science, 1(2):2793–2800, 2010.

[26] K. A. Smith, M. Dennis, and J. Masthoff.

Personalizing reminders to personality for melanoma self-checking.Proceedings of the 24th International Conference on User Modeling, Adaptation and Personalization, In Press.

[27] K. A. Smith, M. Dennis, J. Masthoff, and N. Tintarev.

A Method of Bootstrapping Adapation using Personality Stories, In Progress.

[28] N. Soonthornphisaj, E. Rojsattarat, and

S. Yim-Ngam. Smart e-learning using recommender system. InComputational intelligence, pages 518–523.

Springer, 2006.

[29] W. L. Taylor. Cloze procedure: A new tool for measuring readability.Journalism Quarterly, 30:415–433, 1953.

[30] N. Tintarev, M. Dennis, and J. Masthoff. Adapting recommendation diversity to openness to experience:

A study of human behaviour.Proceedings of the 21th International Conference on User Modeling,

Adaptation, and Personalization, pages 190–202, 2013.

[31] N. Tintarev and J. Masthoff.Recommender Systems Handbook, chapter Explaining Recommendations:

Design and Evaluation, pages 353–382. Springer US, Boston, MA, 2015.