Undergraduates Jenna Snyder and Rachel Casper studied the differences in how composition and TESOL teachers grade essays written by non-native English speakers.
Are teachers biased when it comes to grading non-native speakers? Composition teachers in the U.S. generally expect to teach native English speakers who are proficient in English and familiar with American culture. TESOL (Teaching English to Speakers of Other Languages) teachers, however, assume their students are less familiar with the English language and culture, so instead focus on helping their students gain a better grasp of, and more confidence in, speaking and writing English. This difference in teacher backgrounds may affect the way they see and grade student writing. To discover how these teachers’ grading practices differed when presented with the same stimulus, Grant Eckstein, assistant professor of linguistics, mentored the ORCA research of two undergraduates: Jenna Snyder and Rachel Casper. They found that there were definitely differences in how teachers graded essays based on their collegiate training.
These two undergraduates are particularly interested in second language teaching processes. Snyder recently graduated with a degree in English and a minor in TESOL. Casper is pursuing an undergraduate degree in linguistics and has presented this research locally, regionally, nationally, and internationally. She and Eckstein coauthored a paper (along with two other undergraduates [Madeline Noxon and Wesley Schramm]) which is currently under review with an academic journal.
To measure what composition and TESOL teachers looked at longest while grading an essay, Eckstein, Snyder, and Casper used an eye-tracking approach, something that has rarely been used to study grading differences. “The technology uses a near-infrared invisible light. There is a camera placed above the screen where the teacher is reading which tracks the reflection of the infrared light on the eye,” explained Snyder. Eyeglasses and even contacts can interfere with eye-tracking technology and partially for this reason, the sample size was small, the group testing five composition teachers and five TESOL teachers. Each of the teachers was a current or former BYU Master’s student who had around five semesters, on average, of university teaching experience.
The researchers’ process also involved interviews of the teachers before and after they were presented with the essays they would be grading. In the interviews, both composition and TESOL teachers admitted they approached the grading of L1 essays (written by native English speaking students) and L2 essays (written by non-native speaking students) differently, but both thought they would be more lenient of grammar errors by the L2 writers. The L1 and L2 essays for this study both came from first-year composition students at a university in the western U.S. who were asked to write about their background and process as a writer. The L2 essay was written by a native Mandarin-speaking student in the same course who had approximately 13 years of prior English study including one developmental writing course in college.
In the eye-tracking study, each teacher was presented with a rubric to use while grading the essays. Most of the teachers had received training in rubric creation and used rubrics in their own classes, so the structure was familiar. To eliminate as much bias as possible from the test, the teachers were not informed of the nationality, background, or linguistic status of each essay writer. “Teachers have a number of biases which affect how they read a text,” explained Casper. “One specifically is an ethnolinguistic bias. Even seeing a student’s name can affect a teacher’s reading of the text.” This was especially crucial for this study because many of the composition teachers had reported never teaching an L2 student in their classes.
From the beginning, however, it was fairly easy to tell which essay was written by a native English speaker and which was not. Many TESOL instructors teach the five paragraph essay format as a simple way to organize writing. The word choice in the L2 essay was also more repetitive and contradictory. The L1 essay showed a much different structure, with a different organizational arrangement (i.e., only two paragraphs) and more variety in word choice and phrasing.
Looking at the results, the researchers found that the teachers did exhibit several differences in grading. Composition teachers tended to focus on points of the essay associated with organization and word choice, while the TESOL teachers spent more time looking at rhetorical features and grammar. Overall, composition teachers gave both essays a lower score than TESOL teachers, though both groups of teachers rated the L2 essay below the L1. The main difference, however, appeared in what teachers spent time looking at. Composition teachers spent more time reading the organizational features of the L1 essay, looking more closely at the structure of writing than in the L2 essay, presumably because the L1 organization was more sophisticated. The TESOL teachers spent more time looking at the grammatical errors of the L1 essay, possibly because they were less common errors than TESOL teachers usually encounter. Finally, composition teachers tended to read and grade the L2 essay more linearly by reading it holistically while the TESOL teachers tended to backtrack, re-read, and confirm their reading more frequently. In short, the teachers had different grading approaches, looked at different aspects of the texts, and ended up assigning slightly different scores.
As globalization of education and business continues, understanding how evaluators read and judge writing—particularly L2 writing—is becoming increasingly important. Though the sample size in this study is too small to make sweeping generalizations, Casper hopes that further research will lead to more dialogue among composition and TESOL writing teachers. “They come from very different trainings in language teaching and there’s a lot to be learned from one another,” she said.
—Hannah Sandorf Davis (B.A. Art History and Curatorial Studies ’17)
Hannah covers events for the linguistics department of the College of Humanities. She is a senior pursuing a degree in art history with a minor in art.
Image of Rachel Casper