“A change in Quantity also entails a change in Quality.”
–ENGELS FRIEDRICH
The aim of the dissertation is to investigate the acquisition of L2 English articles and countable/uncountable nouns in L1 speakers of Mandarin. The dissertation seeks to understand how the use of linguistically-informed teaching materials (LIMs) in explicit instruction—designed in the framework of Generative Second Language Acquisition (GenSLA)—assists in the acquisition of these phenomena in comparison to the use of traditional teaching materials. This chapter begins with an overview of the general research design. Following that, the participant groups will be discussed—their level of English, biodata and language use information, and their experience in an instructed SLA setting. Section 6.3 discusses each of the data collection instruments. In the Section 6.4, we will present the general research procedures, including research ethics approval, the pilot phase, the experimental data collection phase and the motivation and implementation of the intervention materials. In the final part of the chapter, the research questions (RQs) and hypotheses that were presented in Chapter 1 are revisited.
6.1 | General Research Design
The research design uses a pre-test/post-test design, with the same tasks (with different stimuli) being completed by the participants at all data collection times. As can be seen in Table 6.1, the experiment took place over a five-week period. Prior to the pre-test (T0), the participants received consent materials and completed pre-participation questionnaires. A copy of the consent materials can be found in Appendix A, and copies of the two pre-participation questionnaires can be found in Appendix B. The pre-test (T0) data was collected during week 1. The following week, week 2,
the participants receiving additional instruction for the treatment attended the instructional intervention session prior to post-test 1 (T1) data collection. The intervention was presented as a
“grammar workshop”, during which one of the participant groups were taught about English articles and noun type distinctions vis-à-vis the use of semantic universals and a linguistic analysis of their native language (L1) and how it compares to English, henceforth linguistically-informed materials (LIMs). For another group, the treatment consisted of additional English article and noun type instruction vis-à-vis the use of their curriculum-assigned grammar textbook. The treatment session lasted one hour. A control group was also included in the study and did not receive any form of instructional intervention. After the instructional intervention, the participants had a small break before completing the immediate post-treatment experimental data collection (T1). Three weeks later, the participants returned to the same lab to complete the delayed post-test, post-test 2 (T2) data collection tasks.
Table 6.1. Research design and data collection times for all non-native speaker (NNS) participant groups Before
Each data collection session consisted of three tasks. The three data collection tasks tapped into implicit knowledge and explicit knowledge, an elicited-sentence imitation task, an acceptability judgment task, and a forced-choice elicitation task. According to R. Ellis (2009b), it is “possible to develop [tasks] that will provide relatively separate measures of implicit and explicit knowledge”
(p. 169). As to prevent the participants from noticing what linguistic features were being tested in the study’s design, the more implicit task was always administered first, and the more explicit tasks followed. All three tasks were completed at each data collection time. Section 6.3 describes each of the data collection instruments in detail.
Prior to the experimental data collection, a small pilot study was completed with a small group of NSs of Mandarin (n = 7). This pilot study took place in on-campus computer labs at one of the same Midwestern United States universities as the experimental data collection later did. The pilot
study revealed some methodological and technological concerns. Each of the methodological concerns will be discussed in their respective data collection tasks later in the chapter.
6.2 | Participants
The experimental participants in the study were 78 college-aged (17-30) L1-Mandarin learners of L2 English at a public university in the Midwestern United States. The participants were in English as a Second Language (ESL) courses in a credit-bearing program. The participants were receiving formal grammar instruction (3 hours a week), as well as formal instruction in a variety of other ESL courses (oral skills, reading, writing, and listening) for up to an additional 9 hours of ESL instruction each week. These participants were divided into three groups that received different interventions: (i) LING: those who received additional linguistically-informed instruction (n = 30), (ii) TRAD: those who received additional traditional instruction (n = 18), and (iii) NOEX: those who did not receive any additional hours of instruction (n = 17). All of these participants came from the same university-sanctioned credit-bearing ESL program. While all participants were enrolled in an ESL-credit program, they had the option to receive additional course credit for their participation in the study. The native speaker (NS) baseline group (n = 25), L1-English speakers (ENGL), were recruited from a general education course in a linguistics department at the same public university in the Midwestern United States. Similarly, these participants were also offered course credit for their participation in the study. We now describe each of the participant groups in detail.
Experimental Group: Linguistically-Informed Instruction
The first experimental group of participants (n = 30) included 4 men and 21 women between the ages of 19 and 23 years (M = 20.4, SD = 1.12), all of whom were L1 Mandarin speakers who had come to the United States to pursue their undergraduate degrees; had at least 12 years of education in primary and secondary school prior to arrival in the USA; and had enrolled in one or more courses of formal ESL instruction during their time of their participation in the study. In order to establish a homogenous group since the participants did not come as a cohort from a pre-established language courses, they completed the Oxford Quick Placement Test (OQPT) prior to their participation in the study with scores out of 60 ranging between 20 and 54 (M = 35.9, SD = 6.13). This participant group received one hour of linguistically-informed instruction between testing times T0 and T1. Henceforth, this group will be referred to as LING. Their particular
Experimental Group: Traditional Instruction
The second experimental group of participants (TRAD) (n = 18) included 9 men and 9 women between the ages of 17 and 24 years (M = 19.39, SD = 1.29), all of whom were L1 Mandarin speakers who had come to the United States to pursue their undergraduate degrees; had at least 12 years of education in primary and secondary school prior to arrival in the USA; and had enrolled in one or more courses of formal ESL instruction during their time of their participation in the study. In order to establish a homogenous group since the participants did not come as a cohort from a pre-established language courses, they completed the OQPT prior to their participation in the study with scores out of 60 ranging between 19 and 58 (M = 33.94, SD = 9.53). This participant group received instruction using their regular ESL textbook as their 1-hour intervention between testing times T0 and T1. This group will be referred to as TRAD for the remainder of the dissertation. The intervention for this particular group will be discussed in more detail in Section 6.5.1.
Experimental Group: No Extra Instruction
The third experimental group of participants (n = 17) included 5 men and 12 women between the ages of 18 and 25 years (M = 19.47, SD = 1.87), all of whom were L1 Mandarin speakers who had come to the United States to pursue their undergraduate degrees; had at least 12 years of education in primary and secondary school prior to arrival in the USA and had enrolled in one or more courses of formal ESL instruction during their time of their participation in the study. In order to establish a homogenous group since the participants did not come as a cohort from a pre-established language courses, they completed the OQPT prior to their participation in the study with scores out of 60 ranging between 13 and 45 (M = 31.94, SD = 7.53). This participant group did not receive any intervention between testing times T0 and T1.
Baseline Group: Native Speakers of English
The fourth group of participants (n = 25) was a group of undergraduate native speakers of English which included 4 men and 21 women between the ages of 19 and 23 years (M = 20.40, SD = 1.12), all of whom were native speakers (NSs) of English who had come to this university to pursue their undergraduate degrees and had at least 12 years of education in primary and secondary school prior to their university admittance. These participants were recruited from a general education university course, which is a course that is typically taken in the first year or two of their college
educations. This course is required to meet the basic requirements of any bachelor’s degree in the university. These participants were all degree-seeking students studying at a Midwestern university in the United States, and they all spoke American English natively. Although they spoke various dialects of American English, this did not contribute to any significant differences in the collection of the baseline data. Although these participants came from a convenience sample (Plonsky, 2013)—students in their first two years of university in a required general education course—they represented a vast majority of NSs at this age level who are enrolled in degree-seeking programs.
For the NSs, assignment to a specific set of tasks (T0, T1, or T2) happened randomly so that each set of tasks received nearly the same number of participants.
6.3 | Data Collection Instruments
As aforementioned, there were three data collection times. All participants completed three data collection tasks at each data collection time. In the following sections, each of the data collection instruments (participation and data collection) are discussed in detail. Descriptions of the pre-participation questionnaires are given first, followed by information about the data collection instruments in their pre- and post-pilot versions. It is important to note that the data collection instruments are presented in the same order in which the participants completed them.
Biodata & Language Use Questionnaire
Prior to participation in the study, the participants were asked to complete two pre-participation questionnaires to demonstrate their willingness to participate in the study. The first pre-participation questionnaire was the Biodata and Language Use Questionnaire (BLUQ), adapted from Keijzer (2007), that was programmed into an online survey software, Qualtrics (Qualtrics, 2017). This questionnaire consisted of approximately 60 questions that tapped into the participants’ biographical data, information about their native language, experience with the second language, and some questions pertaining to their length of residency in the USA and their exposure to ESL. This data was collected to ensure that the participant groups were as homogenous as possible, as well as representative of L1 Mandarin international students in the United States. There was a separate BLUQ for the NSs. This questionnaire had approximately seven questions. Copies of both version of the BLUQ can be found in Appendix B.
English Proficiency Test
The second pre-participation questionnaire was the Oxford Quick Placement Test (OQPT) (U. C.
L. E. Syndicate, 2001) that had been programmed into Qualtrics (Qualtrics, 2017), an online survey software. The OQPT is a multiple-choice test that assesses the general English proficiency level of students via a variety of reading, vocabulary, and grammar tasks that target a variety of simple and complex grammatical structures, with some questions focusing on meaning and others focusing on specific grammatical forms. This test gives a score of 60 that can then be mapped onto levels of the Common European Framework of Reference (CEFR). A copy of the OQPT can be found in Appendix B.
While the original version of the test consists of two separate parts with 30 questions each, all participants took a version which consisted of all 60 questions. The participants were instructed that they could skip any question that they found too difficult, although this would be calculated into their final score for the test. This test is well-known and used widely as a test of English language proficiency. For this reason, the proficiency level of participants in this study can be directly compared with that of participants in other published studies of the same level. The OQPT scores, along with descriptive statistics of the individual participant groups can be found in Table 6.2.
Table 6.2. Descriptive statistics of the participants in the experimental study
Learner Group L1 Age Gender OQPT acceptability judgment task (AJT), and a forced-choice elicitation task (FCET). All tasks measured
four noun types (countable, uncountable-substance, uncountable-object, and flexible) combined with five article conditions (the + singular, the + plural, a + singular, Ø + singular, and Ø + plural).
Since flexible nouns are grammatical in each of the five article conditions due to their shift in interpretation, the decision was made to include flexible nouns in the experimental tasks but omit them from the data analyses.
As Table 6.3 shows, the AJT and ESIT tested 60 sentences: grammatical and 21 ungrammatical statements. There were no fillers because it was determined that the experiments displayed multiple phenomena and the combinations of articles and noun types could not be spotted as the target structure. The uneven number of grammatical and ungrammatical conditions is due to the fact that the “grammatical conditions” were created purely by the article + noun type combinations.
Since not every article + noun combination is even from noun type to noun type, this naturally created an uneven number of grammatical and ungrammatical sentences. The FCET only tested grammatical contexts for each noun, which equated to sentences. An additional 24 distractors testing only article choice and noun type choice brought the total number of items for the FCET to 63.
The reason for selecting three different measures was to gather as much information as possible about the participants’ implicit and explicit knowledge and use of L2 English articles and noun type distinctions. The motivation for the two different types of tasks (implicit versus explicit) was to try and measure the two different types of grammatical knowledge, in order to get an overview of the acquisition of the structures by the participants and to test how explicit instruction affects both of these types of knowledge.
Table 6.3. Number of grammatical and ungrammatical sentences in each version of the ESIT and AJT Noun Type Grammaticality # Sentences Contexts
Countable Nouns grammatical 12
the + singular the + plural a/an + singular Ø + plural
ungrammatical 3 Ø + singular
Uncountable-Object Nouns
grammatical 6 the + singular
Ø + singular
ungrammatical 9
the + plural a/an + singular Ø + plural
Uncountable-Substance Nouns
grammatical 6 the + singular
Ø + singular
ungrammatical 9
the + plural a/an + singular Ø + plural
Flexible Nouns grammatical 15
the + singular the + plural a/an + singular Ø + singular Ø + plural
ungrammatical 0 N/A
Research by R. Ellis (2009a) states that instruments such as the ESIT measure more implicit knowledge and processing, while tasks like an AJT and FCET measure more explicit, metalinguistic knowledge. Accordingly, there is a great deal of research (see R. Ellis et al., 2009)
that does indicate that each of these tasks tap into a different view of the participants’
understanding and knowledge of L2 English articles and noun type combinations.
6.3.3.1 | Elicited Sentence Imitation Task
The first experimental task that all participants completed was a computer-delivered elicited sentence imitation task (ESIT). Similar to acceptability judgment tasks, ESITs are used to determine grammaticality. This task required participants to listen to English grammatical and ungrammatical sentences and provide an oral imitation of those sentences using correct English.
This explored the participants’ implicit knowledge of the article and noun type structures under study. According to Mackey and Gass (2016, p. 65), “The basic assumption underlying elicited imitation is that if a given [structure] is part of one’s grammar, it will be easy to repeat; it is as if sentences are ‘filtered’ through one’s grammatical system”. In this task, sentences were presented auditorily (prerecorded by the researchers), and the participants were asked to repeat them and correct their potential ungrammaticality.
Within the field of SLA, this task has been widely used to elicit data in order to assess the acquisition of L2 oral proficiency (Burger & Chrétien, 2001; among others) and grammar (R. Ellis, 2005b; R. Ellis et al., 2009; R. Ellis, Loewen, & Erlam, 2006; Erlam, 2006; among others). R. Ellis (2005b) performed a study in which he compared the results of four data collection techniques to see if they provided relatively independent measures of implicit and explicit knowledge. R. Ellis (2005b) states that when operationalizing the constructs of L2 implicit and explicit knowledge, the use of metalinguistic knowledge is not required for implicit knowledge, while it is encouraged for explicit knowledge. Furthermore, implicit knowledge is best operationalized when responses are given under a time pressure and the primary focus of the task is meaning.
Although ESITs have been greatly debated, they have been proven to be a valid task in collecting data on a speaker’s implicit knowledge (Erlam, 2009; Spada, Shiu, & Tomita, 2015). As R. Ellis (2005b, p. 45) states, participants’ “failure to imitate a sentence at all or to reproduce it in such a form that they did not create an obligatory context for the target structure of a sentence [is] coded as ‘avoidance’.” If a participant appropriately imitates a sentence where the target structure is correctly supplied, then it is accepted that the form or structure has become part of the learner’s implicit knowledge.
Designing such a task takes careful consideration and time. It is generally agreed that immediate imitation of the sentence should be avoided because it might lead to rote repetition. Our study designed the sentences as belief statements that provided a certain amount of meaning-focus to the participants. The belief statements were then followed by the question “Do you agree or disagree?”, which allowed for the participants to focus on the meaning of the statement, and how they felt about it, rather than the explicit grammaticality or the form of the statements.
Additionally, the optimal time latency between the prompt and the imitation might be influenced by the participants’ proficiency. For this reason, this specific ESIT was designed with a 3000ms delay to avoid it being too short, but also not too long, before prompting for a repetition. A final consideration involved the length of the stimuli, which should not be too short as to be easily memorized, nor should it be too long in which it might go beyond the capacity of short-term memory (Bley-Vroman & Chaudron, 1994). Considering all of the factors mentioned above and following the work by Erlam (2009), this task consisted of 60 belief statements involving both grammatical and ungrammatical sentences containing the target article and noun type combinations. The instructions that the participants were presented with can be found in Figure 6.1.
Figure 6.1. Directions for the ESIT
This task was programmed in Ibex Farm11. The sentences were presented audibly to the participants, who were required to indicate whether they agreed or disagreed with the content of each statement. The only thing visible on the screen while the participants heard the stimuli was
11 Ibex Farm is an open-source online program developed by Alex Drummond. It can be accessed online at http://spellout.net/ibexfarm.
In this experiment, you will hear sentences and be asked whether you agree or disagree with them. After each sentence, you will indicate whether you agree or disagree with the belief statement that you heard. You will only have 3 seconds to record your answer. After recording your answer, the screen will advance, and you will need to repeat the sentence you heard earlier.
You should repeat the sentence in correct English. Listen carefully to each sentence and do not speak until the screen says that you may record your sentence. Speaking before the cue is given will lead to the data not being properly recorded.
The following five sentences are practice items. After the practice items, the experiment will begin.
the symbol “+”. The screen then moved on to a message that prompted the students to agree or
the symbol “+”. The screen then moved on to a message that prompted the students to agree or