Principal component analysis of the main scales

The above analysis showcases the different phases of a concept-based technique. As I demonstrated in the previous chapter, this approach meant that in designing the scales for this study, I first carried out an in-depth literature review that allowed me to select the relevant constructs to be included and then to define them. Finally, the item sets were developed based on these definitions and on existing questionnaires standardized through previous research.

These steps also ensured the external validity of the instrument, as each item in a scale was designed so that the set would correspond to the original definition of the construct it represents.

Moreover, the Chronbach’s alpha values presented above demonstrate the high internal consistency of the final scales.

Nevertheless, in order to put the accuracy of this process to the test, I used principal component analysis (PCA) to identify potential underlying variables. I was interested in examining the performance of the standardized scales (cf. 3.3.3) in the context of the study, and in investigating whether the preset item clusters indeed measured different attitudinal concepts.

In line with traditions in the field of L2 motivation and for increased comparability with previous studies, a varimax rotation was used with Kaiser normalization. Thus, the aim of these tests was merely experimental, although the results, summarized in Table 3.20, revealed a remarkable pattern. In general, most of the scales corresponded to a single component and showed a good fit with the original clustering, while other sets presented a more complex structure. Below, I first discuss the former and conclude with the latter category.

First of all, the variable with the highest internal consistency, as identified by the analysis, was composed of the six items in the Willingness to communicate scale. As I mentioned earlier, this is a highly standardized scale, thus it is not surprising that it attained such high scores both in terms of reliability coefficients and component correlations. The figures displayed above echo the results of the internal reliability measures discussed previously. Consequently, the check can be regarded as successful, the scale stood the test.

Interestingly, the second strongest variable was made up of one half of the scale on traditional native speaker groups, namely the English. The five items show a high level of correlation, which is in line with the results of the reliability analysis. On the other hand, the remaining five statements form the fourth variable in the list, therefore they demonstrate similar levels of importance and correlation.

It is not entirely surprising that the scales on motivated learning behavior and attitudes to learning English converge into a single component, given their strong relationship in this (Chapter 4) and previous studies (2.2.3). They each contribute three items to the third factor.

Moreover, this component also includes the first item of the ideal L2 self scale, which was omitted from that set due to its low performance.

The central constructs of the L2 Motivational Self System, the ideal L2 self and the ought-to L2 self form two distinct components, number 5 and 10 respectively. First, this further confirms that the two scales measure two separate concepts, distinct not only from one another but also from the rest of the motivational factors. As mentioned earlier, the first item in the ideal L2 self scale represents the only exception, but, in line with a concept-based methodology followed in the study, that statement was not included in the final analysis. Secondly, the difference in strength between the two scales reflects the issues of researchability (2.2.3) and internal reliability (3.4.3) discussed above.

Table 3.20 Principal Component Analysis (Rotated Factor Matrix^a) of the main attitudinal scales Latent dimensions

8 .026 .084 .250 .678 .073 -.124 .295 -.066 .090 .064 -.171 .143 9 .093 .829 .112 .081 .079 -.098 .106 .038 .022 .091 .096 .003 10 .004 .104 .049 .831 .051 -.092 .033 .018 -.001 .134 .136 -.028 Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 14 iterations.

The central constructs of the L2 Motivational Self System, the ideal L2 self and the ought-to L2 self form two distinct components, number 5 and 10 respectively. First, this further confirms that the two scales measure two separate concepts, distinct not only from one another but also from the rest of the motivational factors. As mentioned earlier, the first item in the ideal L2 self scale represents the only exception, but, in line with a concept-based methodology followed in the study, that statement was not included in the final analysis. Secondly, the difference in strength between the two scales reflects the issues of researchability (2.2.3) and internal reliability (3.4.3) discussed above.

Another standardized scale, the one on ethnocentrism, was identified as the sixth factor. What is more surprising, the items on world citizenship or the global village comprised a single component, the strength of which was ranked seventh in the list. These results suggest that the concept is indeed a valid one and distinct from the other motivational constructs included in the study.

Similarly to the fifth component, which was composed of all but one item in the scale on the ideal L2 self, factor number eight included all except one of the items in the perceived importance set. The third item, which was identified as a weak performer in the reliability tests and was thus discarded from the analysis, formed a scale on its own.

However, there was one scale that did not conform to this pattern, interacting in more complex ways with the other factors. True, similar behavior could be observed when examining the first item of the ideal L2 self scale, which showed more resemblance to the items in the third component, which grouped together MLB and learning attitudes. Nevertheless, the rest of the items in that scale were shown to belong to the same factor. In contrast, the items on international posture almost all contributed to different scales. The first and the last statement together formed factor 9, while second item correlated with the component containing statements on the global village and the third item formed a scale on its own. These results raise questions concerning the applicability of the concept of international posture in the Geneva setting. Part of a standardized scale, the items obtained a combined Cronbach’s Alpha of α = .736, to which all the items contributed. Therefore, the results suggest that there is room

for development through further research, so as to develop this item set into a scale that performs with higher efficiency in the Geneva setting.

In sum, the ten scales formed twelve factors, with the last two consisting of a single item each.

The first ten were thus composed by the joining of two scales, the items on motivated learning behavior and on attitudes to learning English merging into a single component. Unsurprisingly, the scale on attitudes to target groups clearly revealed its double composition. Overall, as the analysis indicates, most of the item sets performed well as individual variables.

Although the PCA results did not impact further procedures, they allowed a few interesting observations. First, as one of the goals of the study was to test the standardized scales in the Geneva context, it can be concluded that the dataset examined confirmed the integrity and validity of the item sets. It is also to be noted that the same item set might produce different components in different settings, and, as a consequence, differences between contexts might be expressed not only at the level of motivational factors but also in the internal structure of these elements. While the above can provide some pointers as to the particular sample at hand, further research at a larger scale is needed to explore this angle of potential contextual influences.

3.5 Piloting

The paper-based questionnaire was piloted through a series of think aloud sessions. Due to the considerable inaccessibility of the target population, participants of the pilot sessions were selected from among the researcher’s acquaintance in order to match the target population. Five students from different neighboring universities took part in the sessions, two of them male and three female. Their areas of study ranged from biology to medicine to economics, and they represented all three levels of university education, from bachelor to PhD, although no specialized master’s students took part in the pilots.

During each session of 30 to 40 minutes, one or two students were asked to fill out the paper-based questionnaire and either voice their thoughts or simply mark items for a follow-up discussion. The second option was markedly more popular as a result of the high cognitive load of the task, as most of the participants found it difficult to talk while reading the items and contemplating their answers. Completion of the questionnaire took about twenty minutes in each case, a little less for those who opted to discuss the items separately.

Pilot participants’ comments touched on various aspects of the questionnaire, however, three main themes emerged. First of all, many of their suggestions focused on the wording of the items and they were willing to offer translations that they considered as more appropriate to the context. Secondly, they voiced concerns over the repetitiveness of some of the questions, despite the fact that items appeared in a completely randomized order in the third part of the questionnaire. Last but not least, they pointed out that some of the instructions and phrases required further clarifications, which could be added in the form of footnotes or side notes.

Their comments were taken into account in the subsequent stages of development and contributed greatly to the final digital instrument presented below.