CALICO Journal, Vol 34, No 3 (2017)

The Short-Term and Long-Term Effects of AWE Feedback on ESL Students’ Development of Grammatical Accuracy

Zhi Li, Hui-Hsien Feng, Aysel Saricaoglu
Issued Date: 24 Aug 2017


This classroom-based study employs a mixed-methods approach to exploring both short-term and long-term effects of Criterion feedback on ESL students’ development of grammatical accuracy. The results of multilevel growth modeling indicate that Criterion feedback helps students in both intermediate-high and advanced-low levels reduce errors in eight out of nine categories from first drafts to final drafts within the same papers (short-term effects). However, there is only one error reduction of statistical significance in the category of Run-on Sentence from the first drafts of the first paper to the first drafts of the subsequent papers for both levels of students (long-term effects). The findings from interviews with the participants reveal students’ perceptions of Criterion feedback and help us understand the feedback effect. Implications for a more effective use of AWE tools in ESL classrooms are discussed.

Download Media

PDF Subscribers Only

DOI: 10.1558/cj.26382


Bazeley, P. (2007). Qualitative data analysis with NVivo. Thousand Oaks, California: SAGE Publications.

Bickel, R. (2007). Multilevel analysis for applied research: It’s just regression! New York: The Guilford Press.

Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing, 12(3), 267–296.

Chapelle, C. A. (2003). English language learning and technology: Lectures on applied linguistics in the age of information and communication. Amsterdam: John Benjamin Publishing Company.

Chen, H. J., Chiu, T. L., & Liao, P. (2009). Analyzing the grammar feedback of two automated writing evaluation systems: My Access and Criterion. English Teaching and Learning, 33(2), 1–43.

Chen, C. F., & Cheng, W. Y. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2), 94–112.

Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Los Angeles: SAGE Publications.

Dikli, S., & Bleyle, S. (2014). Automated essay scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17.

Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2), 121–142.

Elliot, N., Gere, A. R., Gibson, G., Toth, C., Whithaus, C., & Presswood, A. (2013). Uses and limitations of automated writing evaluation software. WPA-CompPile Research Bibliographies, 23. Retrieved from

Ellis, R. (2008). A typology of written corrective feedback types. ELT Journal, 63(2), 97–107.

Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A Multi-Site case study of automated writing evaluation. Journal of Technology, Learning, and Assessment, 8(6). Retrieved from

Ferris, D. R. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 81–104). Cambridge: Cambridge University Press.

Ferris, D. R. (2012). Technology and corrective feedback for L2 writers: Principles, practices, and problems. In G. Kessler, A. Oskoz, and I. Elola (Eds.), Technology across writing contexts and tasks. CALICO Monograph. San Marcos, TX: CALICO.

Hayes, A. F. (2006). A primer on multilevel modeling. Human Communication Research, 32, 385–410.

Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39, 83–101.

IBM Corp. (2012). IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.

Kennedy, S. (2010). Corrective feedback for learners of varied proficiency levels: A teacher’s choices. TESL Canada Journal, 27(2), 31–50.

Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18.

Li, S. (2010). The effectiveness of corrective feedback in SLA: A meta-analysis. Language Learning, 60(2), 309–365.

Li, Z., Link, S., Ma, H., Yang, H., & Hegelheimer, V. (2014). The role of automated writing evaluation holistic scores in the ESL classroom. System, 44, 66–78.

Link, S., Dursun, A., Karakaya, K., & Hegelheimer, V. (2014). Towards better ESL practices for implementing automated writing evaluation. CALICO Journal, 31(3), 323–344.

Otoshi, J. (2005). An analysis of the use of Criterion in a writing classroom. The JALT CALL Journal, 1(1), 30–38.

Rich, C. S. (2012). The impact of online automated writing evaluation: A case study from Dalian. Chinese Journal of Applied Linguistics, 35(1), 63–79.

Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65.

Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics (4th ed.). Boston: Allyn and Bacon.

Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46, 327–369.

Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 16(4), 255–272.

Wang, P. (2013). Can automated writing evaluation programs help students improve their English writing? International Journal of Applied Linguistics & English Literature, 2(1), 6–12.

Wang, Y.-J., Shang, H.-F., & Briody, P. (2013). Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Computer Assisted Language Learning, 26(3), 234–257.

Ware, P., & Hellmich, E. (2014). CALL in the K–12 context: Language learning outcomes and opportunities. CALICO Journal, 31(2), 140–157.


  • There are currently no refbacks.

Equinox Publishing Ltd - 415 The Workstation 15 Paternoster Row, Sheffield, S1 2BX United Kingdom
Telephone: +44 (0)114 221-0285 - Email:

Privacy Policy