Using Grammar Checkers in an ESL Context
An Investigation of Automatic Corrective Feedback
DOI:
https://doi.org/10.1558/cj.36523Keywords:
grammar checkers, written corrective feedback, focus on form, English second language learningAbstract
Our study examines written corrective feedback generated by two online grammar checkers (GCs), Grammarly and Virtual Writing Tutor, and by the grammar checking function of Microsoft Word. We tested the technology on a wide range of grammatical error types from two sources: a set of authentic ESL compositions and a series of simple sentences we generated ourselves. The GCs were evaluated in terms of (1) coverage (number of errors flagged), (2) appropriacy of proposed replacement forms, and (3) rates of “false alarms” (forms mistakenly flagged as incorrect). Although Grammarly and Virtual Writing Tutor outperformed Microsoft Word, neither of the online GCs had high rates of overall coverage (<50%). Consequently, they cannot be relied on to supply comprehensive feedback on student compositions. The finding of higher identification rates for errors from simple rather than authentic sentences reinforces this conclusion. Nonetheless, since few inaccurate replacement forms and false alarms were observed, only rarely is the feedback actively misleading. In addition, the GCs were better at handling some error types than others. Ultimately, we suggest that teachers use GCs with specially designed classroom activities that target selected error types before learners apply the technology to their own writing.
References
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17, 102–118. https://doi.org/10.1016/j.jslw.2007.11.004
Bitchener, J., & Knoch, U. (2010). Raising the linguistic accuracy level of advanced L2 writers with written corrective feedback. Journal of Second Language Writing, 19, 207–217. https://doi.org/10.1016/j.jslw.2010.10.002
Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing, 12, 267–296. https://doi.org/10.1016/S1060-3743(03)00038-9
Chen, H.-J. H. (2009). Evaluating two web-based grammar checkers – Microsoft ESL Assistant and NTNU statistical grammar checker. Computational Linguistics and Chinese Language Processing, 14(2), 161–180.
Chodorow, M., Tetreault, J. R., & Han, N.-R. (2007). Detection of Grammatical Errors Involving Prepositions. In F. Costello, J. Kelleher and M. Volk (Eds.), Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions (pp. 25–30). https://doi.org/10.3115/1654629.1654635
De Felice, R., & Pulman, S. G. (2008). A classifier-based approach to preposition and determiner error correction in L2 English. In D. Scott and H. Uszkoreit (Eds.), Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) (pp. 169–176). https://doi.org/10.3115/1599081.1599103
Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006
Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1, 3–18. https://doi.org/10.5070/L2.V1I1.9054
Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and unfocused written corrective feedback in an English as a foreign language context. System, 36, 353–371. https://doi.org/10.1016/j.system.2008.02.001
Feng, H.-H., Saricaoglu, A., & Chukharev-Hudilainen, E. (2016). Automated error detection for developing grammar proficiency of ESL learners. CALICO Journal, 33(1), 49–70. https://doi.org/10.1558/cj.v33i1.26507
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 81–104). Cambridge, England: Cambridge University Press. https://doi.org/10.1017/CBO9781139524742.007
Ferris, D. (2011). Treatment of error in second language writing (2nd ed.). Ann Arbor, MI: University of Michigan Press. https://doi.org/10.3998/mpub.2173290
Ferris, D., Liu, H., Sinha, A., & Senna, M. (2013). Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22, 307–329. https://doi.org/10.1016/j.jslw.2012.09.009
Gamon, M., Leacock, C., Brockett, C., Dolan, W. B., Gao, J., Belenko, D., & Klementiev, A. (2009). Using statistical techniques and web search to correct ESL errors. CALICO Journal, 26(3), 491–511. https://doi.org/10.1558/cj.v26i3.491-511
Granger, S. (2003). Error-tagged learner corpora and CALL: A promising synergy. CALICO Journal, 20(3), 465–480.
Guénette, D. (2007). Is feedback pedagogically correct? Research design issues in studies of feedback on writing. Journal of Second Language Writing, 16, 40–53. https://doi.org/10.1016/j.jslw.2007.01.001
Han, N.-R., Chodorow, M., & Leacock, C. (2006). Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12(2), 115–129. https://doi.org/10.1017/S1351324906004190
Hoang, G. T. L., & Kunnan, A. J. (2016). Automated essay evaluation for English language learners: A case study of MY Access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121
Jurma, W. E., & Deidre, L. F. (1984). Effects of immediate instructor feedback on group discussion participants. Central States Speech Journal, 35(3), 178–86. https://doi.org/10.1080/10510978409368186
Lavolette, E., Polio, C., & Kahng, J. (2014). The accuracy of computer-assisted feedback and students’ responses to it. Language Learning & Technology, 19(2), 50–68.
Liu, S., & Kunnan, A. J. (2016). Investigating the application of automated writing evaluation to Chinese undergraduate English majors: A case study of WriteToLearn. CALICO Journal, 33(1), 71–91.
Samuels, S. J., & Wu, Y. (2003). The effects of immediate feedback on reading achievement. Technical report. Minneapolis: University of Minnesota. Retrieved from http://www.epsteineducation.com/home/articles/file/research/immediate_feedback.pdf
Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on learners’ accuracy in using two English grammatical structures. Language Learning, 64, 103–131. https://doi.org/10.1111/lang.12029
Tetreault, J. R., & Chodorow, M. (2008). The ups and downs of preposition error detection in ESL writing. In D. Scott and H. Uszkoreit (Eds.), Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) (pp. 865–872). https://doi.org/10.3115/1599081.1599190
Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 16(4), 255–272. https://doi.org/10.1016/j.jslw.2007.06.003
Yi, X., Gao, J., & Dolan, W. B. (2008). A web-based English proofing system for English as a second language users. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP 2008) (pp. 619–624).
