Kara Johnson
Fall 2008

LRC530 Research Proposal

Using computer-searchable texts (e.g. corpora of academic writing, PDF formatted literature) to help students self-correct formerly considered “untreatable” errors, specifically word order errors.

Introduction

There is some debate among the literature as to whether or not correcting errors in academic writing is actually beneficial to second language learners, and if it is, what sort of correction is helpful.  Ferris (1999) strongly disputes Truscott’s (1996) argument that second language learners do not benefit from teachers correcting their written errors.  Her own research, which included a better experimental setup with a control group not receiving any feedback, showed results that the two groups receiving different types of corrective feedback both showed improvement significantly more than the group not receiving any feedback (Ferris & Roberts 2001).

Considering that students often indicate that they want feedback and that it has been shown that some feedback helps them correct, researchers and teachers have considered how to best address their written errors and give feedback.  Teachers vary in their beliefs of how explicit their feedback to errors should be, and their practices of how they address the errors (whether due to time factors or belief factors).  Most researchers agree that there is a need for students’ attention to be brought to the problem, either to the grammatical error or to the problem it has in communicating its intended meaning (Truscott, 1999; Ferris, 1999; Ferris & Roberts, 2001).

Ferris (1999) suggests that, in general, students can, and should, be taught to self-correct.  To be successful in this self-correction, she suggests that they need to be trained to identify and correct patterns of frequent and serious errors and should be given explicit teaching as needed about the rules governing these patterns of errors” (p. 5).  Along with this, she favors teachers helping the students to identify that an error has occurred over the teacher directly correcting the students’ errors.  She states that the question is not whether error correction should be done, but rather how it should be done. 

Research has been done in attempts to determine how to best correct errors in second language students’ writing.  Ferris (1999) argues that how the error is treated should depend on the type of error.  In general, she suggests that students can, and should, be taught to self-correct.  In a study of twenty-one university ESL students at the junior level, she found that fifty percent of the errors that occurred—such as subject-verb agreement, run-ons and comma splices, missing articles, and verb form errors—were “treatable”:  able to be corrected by following a system of rules. For these errors, merely identifying where the error occurs (e.g. underlining) is often sufficient. The other fifty percent of the errors included “missing words, unnecessary words, and word order problems” which were not “treatable,” by her definition. 

Statement of the problem

It has been shown that many students benefit from correcting their own errors, rather than a teacher marking the corrections, but it has also been found that as much as 50% of the errors made by students are “not treatable” (Ferris, 1999).  Because of this, it has assumed that teachers must explicitly correct these “untreatable” errors because the student would have no other “systemic” way of correcting the errors themselves.  If students cannot self-correct by following simply stated grammar rules—but self-correction is the objective—teachers should consider new technology that is becoming more widely available.  One such technology is linguistic corpora (collections of written and spoken language used in “real life” situation”) that is now available online to many teachers, and it may provide a way to guide students in self-correcting these un-systemic types of errors.  Searchable corpora allow the user to enter a word or phrase and search for “real” example of its use.  This can allow students to notice patterns in the language where there are not easily identifiable rules which may help them to self-correct and perhaps also help them develop a better intuitive sense of the language.

Missing words, unnecessary words, inappropriate or inaccurate words, and word order problems would all be this type of “untreatable” error, but this study will focus only on “word order” errors.  The reason for choosing this particular error is that with the words present (but in the wrong order), students, with some training in searching corpora, are likely to be able to find correct examples of the sentence structure in which the words normally appear.  If the study shows that students are able to self-correct from the examples, it may be worth further investigating the other non-systemic types of errors.

Research Questions

1)      If the teacher highlights errors that are not systemic (cannot be corrected by a simple system of rules; e.g. “missing words, unnecessary words, and word order problems”), could a search of similar occurrences (occurring correctly) in corpora or PDF formatted articles or literature help the student correct their errors on their own? 

2)      Is there more benefit for higher or lower proficiency students (considering intermediate and advanced proficiencies)?

3)      Are there long term benefits to those students’ writing or ability to self-correct?

Participants

Because of the nature of corpora searches and the level of proficiency needed to comprehend the search results, intermediate to advanced language learners will be recruited.  The study could be conducted on a university campus in the US, such as the University of Arizona, but I propose to conduct my study in China at the Nanjing University in order to consider the use of this method of error correction in a non-native speaking environment.  Corpora with a large number of examples of native speaker language usage could be particularly helpful in such an environment.

A new program of teaching English composition courses (UofA’s ENGL106, 107 and 108) at Nanjing University was started this academic year, and I propose to conduct the study with these Chinese students.  Two levels of classes would run concurrently (ENGL106—intermediate proficiency, and ENGL107—advanced proficiency), so this would allow me to consider my second research question regarding the benefit for different levels of proficiency.

Research Design and Factors

The study will look at the number of errors that are “wrong word order” (and cannot be corrected by a simple system of rules) made in each sample of writing.  These samples will be drafts of the essays that the students write for the class and submit to the teacher and that the teacher returns with “word order” errors highlighted.  The number of uncorrected or wrongly corrected errors will be compared with the number that are self-corrected. 

The variables will be the students’ level of English (intermediate and advanced proficiencies) and the success rate of corrections (corrections divided by the total number of errors).  All participants will receive their papers marked with “word order” errors, and within each proficiency level, one group of students will receive training on correcting using the corpora and the other group will not be instructed on a particular correction method but will correct it by their own methods.

The success rates of the corrections will be compared to recorded “think aloud” comments of randomly selected participants as they make the corrections and search for an appropriate phrase in the corpora.  This may reveal if there is a systematic way in which the students successfully made the corrections and to determine if, for these students, if word order is an error “treatable” by this method of using corpora searches.  It may also be helpful to note how the students attempt to make the corrections in which the result is still not correct in order to determine if more or different training would be beneficial.

To determine possible long-term effects of this method of self-correction, “permanent” email addresses will be collected so that some participants can be contacted later in their undergraduate study (approx. 1 year later) to be asked to complete a survey regarding their methods for self-correcting their own writing in courses after this study was completed.  This is in hopes of finding out if they continued to use corpora searches and why, according to their self-reflection, they did or did not.  One year would give them time to have completed their required composition courses but they would still be taking university courses in which they would likely still be writing academic papers.

Procedure and Data Collection

Data will be collected through analyzing students’ essays.  These essays collected will be “final drafts” submitted, as well as a version of this final polished draft corrected for the identified errors.  It is anticipated that even final drafts will contain errors that impede the comprehension and that the students’ grades can be raised by correcting for “word order” errors using corpora or PDF-formatted literature searches.  This process of collecting final and polished drafts will be done for at least three papers written in the course.  For at least one final draft that selected students receive back with errors highlighted, the students will try to correct the highlighted errors while the researcher observes.  The student will narrates out loud (in L1 or L2, as the student prefers) what he is thinking about as he considers why it was highlighted, what he can do to correct it, and what keyword searches in the corpora he should try.  If he finds examples similar to what he wants to say, he will narrate his thoughts in applying that to his own writing.

A survey of students (as described above) before the end of their degree program will be part of the long-term data collection.

The teachers will need to train the students on techniques of searching the corpora, given that the “word order” errors are only highlighted.  Some techniques for searching the corpora can include searching for the phrase incorrectly as written, adjusting the phrase order and searching, or searching for the individual words.

Possible Findings

The results that I hope to find are that the groups who correct by corpora searches will have more accuracy in their corrections than the groups who correct by other methods.  It is also possible from the student narrations that we can learn that students perceive the corpora search results as helpful but perhaps will feel they are too complicated or laborious to want to use on their own.  Results showing an improvement using corpora searches would indicate that Ferris’ (1999) discussion of word order as untreatable has not taken into account other methods of “treating” such errors.  These results would support her claim that “patterns” can treated, only the method of finding those patterns are expanded using corpora.

Alternatively, the results could show that there is no significant improvement to corrections using corpora searches, supporting Ferris’ (1999) that this method of corpora searches are not sufficient for correcting such “untreatable” errors.  If this is the result, it may need to be considered other ways to refine corpora use to better help language learners identify the systemeticity of the sentence structure or word usage.

Teachers often look for ways to help their students independently find their own errors and correct them, so having the extensive body of “real” language examples available using corpora and computer-searchable literature shows promise. Additionally, the results of this study will hopefully generate insights and provide additional resources (corpora and computer-searchable literature) to help both second language textbook designers and language teachers as they develop second language academic writing courses which will provide the students with resources and methods of self-correcting errors.

Bibliography

Ferris, D. (1999). The case for grammar correction in L2 writing classes: A response to Truscott (1996). Journal of Second Language Writing, 8 (1), 1-11.

Ferris, D & Roberts, B. (2001). Error feedback in L2 writing classes: How explicit does it need to be? Journal of Second Language Writing, 10, 161-184.

Truscott, J. (1996). The Case Against Grammar Correction in L2 Writing Classes. Language Learning, 46(2), 327-369.

Truscott, J. (1999). Unconscious Second Language Acquisition: Alive and Well. Studies in English Literature and Linguistics, 25, 115-131.