AI-Generated Feedback in English Writing: Proficiency Development and Learner Perceptions
DOI:
https://doi.org/10.37253/iallteach.v7i2.10503Keywords:
AI-generated Feedback, Teacher Feedback, English Writing Instruction, College StudentsAbstract
This study was situated within the context of college-level English-as-a-foreign- language education in China. It compared the effectiveness of AI-generated feedback and teacher feedback in improving English learners’ writing proficiency. A questionnaire survey and semi-structured interviews were also conducted to examine learners’ acceptance of and attitudes toward AI feedback. Results revealed that both the experimental group (receiving AI feedback) and the control group (receiving teacher feedback) showed significantly higher post-test scores compared to their pretest scores. Although the mean difference in post-test scores between the two groups exceeded that of the pretest, no statistically significant difference was found in their final performance, suggesting comparable efficacy between AI feedback and teacher feedback in enhancing writing proficiency. Additionally, the experimental group demonstrated moderate acceptance of AI feedback, with perceptions varying across proficiency levels. However, learners universally agreed that combining teacher feedback and AI feedback would yield greater benefits for writing development. The study concludes that both feedback modalities have distinct benefits and misfits, advocating for their integrated—rather than exclusive—use in writing instruction.
Downloads
References
Amani, N., & Bisriyah, M. (2025). University Students’ Perceptions of AI-Assisted Writing Tools in Supporting Self-Regulated Writing Practices. IJELTAL: Indonesian Journal of English Language Teaching and Applied Linguistics, 10(1), 91–107. https://ijeltal.org/index.php/ijeltal/article/view/1942
Bai, L., & Hu, G. (2017). In the Face of Fallible AWE Feedback: How do Students Respond?. Educational Psychology, 37(1), 67–81. https://doi.org/10.1080/01443410.2016.1223275
Bai, L, & Wei, Y. (2024). Exploring EFL Learners’ Integration and Perceptions of ChatGPT’s Text Revisions: A Three-Stage Writing Task Study. IEEE Transactions on Learning Technologies, 17, 2161–2172. https://doi.org/10.1109/TLT.2024.3491864
Chen, X. R. (2024). ChatGPT and Second Language Pragmatic Teaching. Foreign Language Learning Theory and Practice, 4, 16–29. https://doi.org/10.13564/j.cnki.issn.1672-9382.2023.03.007
Chiu, T. K., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic Literature Review on Opportunities, Challenges, and Future Research Recommendations of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 4, 100118. https://doi.org/10.1016/j.caeai.2022.100118
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y. S., Gašević, D., & Chen, G. (2023, July). Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 323–325). IEEE. https://doi.org/10.1109/ICALT58122.2023.00100
Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
Deng. J, & Lin, Y. (2022). The Benefits and Challenges of ChatGPT: An Overview. Frontiers in Computing and Intelligent Systems, 2(2): 81–83. https://pdfs.semanticscholar.org/2bf0/2911cbc3bc24e43a910477a90363225bed95.pdf
Ding, L., & Zou, D. (2024). Automated Writing Evaluation Systems: A Systematic Review of Grammarly, Pigai, and Criterion with a Perspective on Future Directions in the Age of Generative Artificial Intelligence. Education and Information Technologies, 29(11), 14151–14203. https://doi.org/10.1007/s10639-023-12402-3
Dong, L. (2024). “Brave New World” or Not?: A Mixed-Methods Study of the Relationship Between Second Language Writing Learners’ Perceptions of ChatGPT, Behaviors of Using ChatGPT, and Writing Proficiency. Current Psychology, 43(21), 19481-19495. http://doi.org/10.1007/s12144-024-05728-9
Elsaiary, R. (2025). Transforming Best Practices and Empowering Educators. In A. ElSayary (Ed.), Prompt Engineering and Generative AI Applications for Teaching and Learning (pp. 161–172). IGI Global Scientific Publishing. http://doi.org/10.4018/979-8-3693-7332-3.ch010
Erisyerico, M. L., & Fauzan, A. (2024). Comparing Students’ Perceptions of AI and Human Feedback in Improving Writing Skills. Scientia, 3(2), 381–387. http://doi.org/10.51773/sssh.v3i2.331
Feng, L. (2025). Investigating the Effects of Artificial Intelligence-Assisted Language Learning Strategies on Cognitive Load and Learning Outcomes: A Comparative Study. Journal of Educational Computing Research, 62(8), 1961–1994. http://doi.org/10.1177/07356331241268349
Godwin-Jones, R. (2022). Partnering with AI: Intelligent Writing Assistance and Instructed Language Learning. Language Learning & Technology, 26(2), 5–24. http://doi.org/10125/73474
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical Versus Mechanical Prediction: A Meta-Analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-3590.12.1.19
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023). A Large-Scale Comparison of Human-Written Versus ChatGPT-Generated Essays. Scientific Reports, 13(1), 18617. https://doi.org/10.1038/s41598-023-45644-9
Huawei, S., & Aryadoust, V. (2023). A Systematic Review of Automated Writing Evaluation Systems. Education and Information Technologies, 28(1), 771–795. https://doi.org/10.1007/s10639-022-11200-7
Hyland, K. (2013). Student Perceptions of Hidden Messages in Teacher Written Feedback. Studies in Educational Evaluation, 39(3), 180–187. https://doi.org/10.1016/j.stueduc.2013.06.003
Ingley, S. J., & Pack, A. (2023). Leveraging AI Tools to Develop the Writer Rather than the Writing. Trends in Ecology & Evolution, 38(9), 785–787. https://doi.org/10.1016/j.tree.2023.05.007
Kline, R. B. (2023). Principles and Practice of Structural Equation Modeling. Guilford Publications.
Kumar, R. (2023). Faculty Members’ Use of Artificial Intelligence to Grade Student Papers: A Case of Implications. International Journal for Educational Integrity, 19(1), 9. https://doi.org/10.1007/s40979-023-00130-7
Leonid, G., Anton, G., & Igor, G. (2024). The Evolution of Artificial Intelligence: From Assistance to Super Mind of Artificial General Intelligence? Article 1. Information Technology and Artificial Intelligence: The Past, Present and Some Forecasts. Social Evolution & History, 23 (1), 156–183. https://doi.org/10.30884/seh/2024.01.07
Li, K. (2023). Determinants of college students’ actual use of AI-based systems: An extension of the technology acceptance model. Sustainability, 15(6), 5221. https://doi.org/10.3390/su15065221
Liu, G., & Ma, C. (2024). Measuring EFL Learners’ Use of ChatGPT in Informal Digital Learning of English Based on the Technology Acceptance Model. Innovation in Language Learning and Teaching, 18(2), 125–138. https://doi.org/10.1080/17501229.2023.2240316
Liu, J., Sihes, A. J. B., & Lu, Y. (2025). How do generative artificial intelligence (AI) tools and large language models (LLMs) influence language learners’ critical thinking in EFL education? A systematic review. Smart Learning Environments, 12(1), 48. https://doi.org/10.1186/s40561-025-00406-0
Mizumoto, A., & Eguchi, M. (2023). Exploring the Potential of Using an AI Language Model for Automated Essay Scoring. Research Methods in Applied Linguistics, 2(2), 100050. https://doi.org/10.1016/j.rmal.2023.100050
Pan, H. Y., & Niu, B. G. (2025). Surveying GenAI-Enhanced Foreign Language Learners’ Writing Feedback Literacy. Technology Enhanced Foreign Language Education, 1, 94–101, 125. http://doi.org/10.20139/j.issn.1001-5795.20250113
Roscoe, R. D., Wilson, J., Johnson, A. C., & Mayra, C. R. (2017). Presentation, Expectations, and Experience: Sources of Student Perceptions of Automated Writing Evaluation. Computers in Human Behavior, 70, 207–221. http://doi.org/10.1016/j.chb.2016.12.076
Shermis, M. D., Burstein, J., & Bursky, S. A. (2013). Introduction to Automated Essay Evaluation In M. D. Shermis & J. Burstein (Eds.), The Handbook of Automated Essay Evaluation: Current Applications and New Directions (pp. 1–15). Routledge.
Tate, T., Doroudi, S., Ritchie, D., Xu, Y., & Warschauer, M. (2023). Educational Research and AI-Generated Writing: Confronting the Coming Tsunami. EdArXiv Preprints. https://doi.org/10.35542/osf.io/4mec3
Tseng, W., & Warschauer, M. (2023). AI-Writing Tools in Education: If You Can’t Beat Them, Join Them. Journal of China Computer-Assisted Language Learning, 3(2), 258–262. https://doi.org/10.1515/jccall-2023-0008
Wang, Y. B., Xuan, Q. Y., & Zhang, J. (2025). The Impact of Generative AI Intervention on Chinese EFL Students’ Engagement with Writing Feedback. Foreign Languages in China, 22(2), 71–78. http://doi.org/10.13564/j.cnki.issn.1672-9382.2025.02.009
Wang, J., & Zhang, X. (2025). Exploring the Role of AI Technology in Shaping College Students’ English Writing Development: Insights from the Complex Dynamic Systems Theory. Journal of Azerbaijan Language and Education Studies, 2(3), 31-46. https://doi.org/10.69760/jales.2025002012
Wei, S., & Li, L. Y. (2023). Artificial Intelligence-Assisted Second Language Writing Feedback: A Case Study of ChatGPT. Foreign Languages in China, 20(3): 33–40. https://doi.org/10.13564/j.cnki.issn.1672-9382.2023.03.007
Weigle, S. C. (2013). English as a second language writing and automated essay evaluation. In M. D. Shermis & J. Burstein (Eds.), The Handbook of Automated Essay Evaluation: Current Applications and New Directions (pp. 36–54). Routledge.
Zhao, Y. (2015). Second Language Creative Writers: Identities and Writing Processes. Multilingual Matters.
Zheng, R. (2025). AWE and LLMs in L2 writing feedback: An exploration of EFL learners’ experiences. Academic Journal of Humanities & Social Sciences, 8(7), 60-69. https://doi.org/10.25236/AJHSS.2025.080708
.png)
