Investigating the Effect of AI Writing Assistance Tools on Iranian Intermediate EFL Learners' Writing Performance: A Comparative Study of ProWritingAid and Grammarly

Subject Areas : Research in English Language Pedagogy

Pourya Borna ¹ , Reza Mohammadi ² , Rahim Karimi Nia ³

1 - Department of English Language, Payame Noor University, Tabriz, Iran
2 - Department of English Language, Payame Noor University, Tabriz, Iran
3 - Department of Language, Tabriz Branch, Islamic Azad University, Tabriz, Iran

Received: 2023-12-13 Accepted : 2024-04-08 Published : 2024-08-11

Keywords: Artificial Intelligence (AI), CALL, Writing Performance, Grammarly, ProWritingAid,

Abstract :

This experiment is intended to determine the consequences of two artificial intelligence writing assistance tools, namely ProWritingAid and Grammarly, on the writing proficiency of Iranian Intermediate EFL learners. Additionally, the study seeks to ascertain which of these tools, ProWritingAid or Grammarly, exerts a more substantial influence on the writing competency of EFL pupils. A quasi-experimental study design was utilized. Through the Oxford Placement Test, a sample of 96 male students was selected from two private language institutes located in Marand, Iran. Two experimental groups and a control group were formed using a non-random allocation method. Each experimental group utilized one of the artificial intelligence writing aid technologies. Subsequently, an eight-session treatment phase was conducted. Then, one-way ANOVA was adopted to evaluate the results of the writing post-test and it pointed out that both experimental groups accomplished exceedingly well compared to the control group. In addition, it was observed that the group that utilized Grammarly exhibited significantly better performance in comparison to the group that used ProWritingAid. The study's conclusions have major ramifications for educators, language learners, and artificial intelligence writing aid technology developers. Educators have the opportunity to employ artificial intelligence driven writing assistance tools for the purpose of delivering prompt and customized feedback to students.

References:

Akhtar, R., Hassan, H., Saidalvi, A., & Hussain, S. (2019). A systematic review of the challenges and solutions of ESL students’ academic writing. International Journal of Engineering and Advanced Technology, 8(5), 1169-1171. http://doi.org/10.35940/ijeat.E1164.0585C19
Akkaya, N., & Kırmızı, F. S. (2010). Relationship between attitudes to reading and time allotted to writing in primary education. Procedia-Social and Behavioral Sciences, 2(2), 4742–4746. https://doi.org/10.1016/j.sbspro.2010.03.761
Alber-Morgan, S., Hessler, T., & Konrad, M. (2007). Teaching writing for keeps. Education and Treatment of Children, 30, 107 - 128. https://doi.org/10.1353/etc.2007.0012.
Andrews, J. M. (2001). Determination of minimum inhibitory concentrations. Journal of antimicrobial Chemotherapy, 48(1), 5-16.
Ariyanto, M. S. A., Mukminatien, N., & Tresnadewi, S. (2019). Students’ and teacher’s perceptions towards the implementation of ProWritingAid and teacher feedback. Jurnal Pendidikan: Teori, Penelitian, dan Pengembangan, 4(10), 1353–1363. http://dx.doi.org/10.17977/jptpp.v4i10.12843
Ayres, R. (2002). Learner attitudes towards the use of CALL. Computer assisted language learning, 15(3), 241–249. https://doi.org/10.1076/call.15.3.241.8189
Azwary, F., F. Indriani and D. T. Nugrahadi. (2016). Question answering system berbasis artificial intelligence markup language sebagai media informasi. Kumpulan Jurnal Ilmu Komputer 3(1), 48–60. http://dx.doi.org/10.20527/klik.v3i1.34
Bonvillain, N. (2019). Language, culture, and communication: The meaning of messages. Rowman & Littlefield.
Chakraverty, A., & Gautum, K. (2000). Dynamics of writing. Forum English Teaching, 38(3), 22-25.
Chang, T. S., Li, Y., Huang, H. W., & Whitfield, B. (2021). Exploring EFL students' writing performance and their acceptance of AI-based automated writing feedback. In 2021 2nd International Conference on Education Development and Studies (pp. 31-35). https://doi.org/10.1145/3459043.3459065
Chastain, K. (1988). Developing second-language skills. Harcourt Brace Jovanovich
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in
field settings. Houghton Mifflin.
Cumming, A. (1998). Theoretical perspectives on writing. Annual Review of Applied Linguistics, 18, 61-78. https://doi.org/10.1017/S0267190500003482.
Daniels, P., & Leslie, D. (2013). Grammar software ready for EFL writers. OnCue Journal, 9(4), 391–401.
Ebadi, S., Gholami, M., & Vakili, S. (2022). Investigating the effects of using Grammarly in EFL writing: the case of articles. Computers in the Schools, 40(1), 85–105. https://doi.org/10.1080/07380569.2022.2150067
Emig, J. (1977). Writing as a mode of learning. College Composition and Communication, 28(2), 122–128. https://doi.org/10.2307/356095
Fahmi, M. A., & Cahyono, B. Y. (2021). EFL students’ perception on the use of Grammarly and teacher feedback. JEES (Journal of English Educators Society), 6(1), 18–25. https://doi.org/10.21070/jees.v6i1.849
Flower, L. (1984). Response to Anthony Petrosky, Review of Linda Flower," Problem-Solving Strategies for Writing.". College Composition and Communication, 35(1), 96-97.
Gunadevi, N., & Narayana, T. (2017). Enhancing of writing skill and common problems. International journal of applied research, 3, 258-261.
Hadiat, A.W.F. (2022). The use of Grammarly to enhance students' accuracy in writing Descriptive text. Journal of English Education Program (JEEP), 9(2), 133–138. http://dx.doi.org/10.25157/(jeep).v9i2.8552
Hamed, J.J. (2012). The effect of peer reviewing on writing apprehension and essay
writing ability of perspective EFL teachers. The Australian Journal of teacher
education, 37(11), 60-84. http://dx.doi.org/10.14221/ajte.2012v37n11.3
Handayani, F., & Handayani, N. D. (2020). The potential of online writing tools for EFL university students during the COVID-19 pandemic. JEE (Journal of English Education), 6(1), 9–18. https://doi.org/10.30606/jee.v6i1.406
Haristiani, N. U. R. I. A., Danuwijaya, A. A., Rifai, M. M., & Sarila, H. (2019). Gengobot: A chatbot-based grammar application on mobile instant messaging as language learning medium. Journal of Engineering Science and Technology, 14(6), 3158–3173.
Hillocks, G. (2002). The testing trap: How state writing assessments control learning. Teachers College Press.
Hwang, G. J., Xie, H., Wah, B. W., & Gaˇsevi´c, D. (2020). Vision, challenges, roles and
research issues of artificial intelligence in education. Computers & Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001
Jacobs, J., E. Zinkgarf, A. Wormuth, S. Hartifel and M. Hughey. (1981). Testing ESL composition: A practical approach. Newbury House.
József, H. (2001). Advanced writing in English as a foreign language. Lingua Franca Csoport.
Kaharuddin, A. (2019). The power of English: Recognizing and utilizing the tremendous impact of the English language on the community. English Language Teaching for EFL Learners, 1(1), 39–48.
Lu, X. (2018). Natural language processing and Intelligent Computer Assisted Language Learning (ICALL). The TESOL encyclopedia of English language teaching, 1–6. https://doi.org/10.1002/9781118784235.eelt0422
Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. Pearson. http://discovery.ucl.ac.uk/1475756/
Matthews, C. (1993). Grammar frameworks in intelligent call. CALICO Journal, 11(1), 5–27. https://doi.org/10.1558/cj.v11i1.5-27
McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence. AI Magazine, 27(4), 12–14. https://doi.org/10.1609/aimag.v27i4.1904
Miranty, D., Widiati, U., Cahyono, B. Y., & Sharif, T. I. S. T. (2021). The effectiveness of using Grammarly in teaching writing among Indonesian undergraduate EFL students. In International seminar on language, education, and culture (ISoLEC 2021) (pp. 41-45). Atlantis Press. https://doi.org/10.2991/assehr.k.211212.008
Myhill, D. (2009). From talking to writing: Linguistic development in writing. The British Psychological Society.
Nilsson, N. J. (2011). The quest for artificial intelligence: A history of ideas and achievements. Cambridge University Press.
https://doi.org/10.1017/CBO9780511819346
Popenici, S. A., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), 1–13. https://doi.org/10.1186/s41039-017-0062-8
Rahma, M., & Zen, E. L. (2023). Errors in students’ writing (Reports from ProWritingAid). Scope: Journal of English Language Teaching, 7(2), 202–209. http://dx.doi.org/10.30998/scope.v7i2.15528
Scardamalia, M., & Bereiter, C. (1982). Assimilative processes in composition planning. Educational psychologist, 17(3), 165-171. https://doi.org/10.1080/00461528209529253.
Shokrpour, N., & Fallahzadeh, M. H. (2007). A survey of the students and interns’ EFL writing problems in Shiraz University of Medical Sciences. Asian EFL Journal, 9(1), 147-163.
Sing, C. C., Teo, T., Huang, F., Chiu, T. K. F., & Xing Wei, W. (2022). Secondary school
students’ intentions to learn AI: Testing moderation effects of readiness, social good and optimism. Educational Technology Research and Development, 70(3), 765–782.
https://doi.org/10.1007/s11423-022-10111-1
Sistani, H., & Tabatabaei, O. (2023). Effects of teacher vs. Grammarly feedback on Iranian EFL learners’ writing skill. International Journal of Foreign Language Teaching and Research, 11(45), 75–87. https://doi.org/10.30495/jfl.2023.703271
Thi, N. K., & Nikolov, M. (2022). How teacher and Grammarly feedback complement one another in Myanmar EFL students’ writing. The Asia-Pacific Education Researcher, 31(6), 767–779. https://doi.org/10.1007/s40299-021-00625-2
Valencia, S. W., & Riddle Buly, M. (2004). Behind test scores: What struggling readers really need? The Australian Journal of Language and Literacy, 27(3), 217-233.
Ventayen, R. J. M., & Orlanda-Ventayen, C. C. (2018). Graduate students’ perspective on the usability of Grammarly® in one ASEAN state university. Asian ESP Journal, 14(7.2).
Wahyuda, M. I. (2022). The effectiveness of Grammarly and ProWritingAid application toward writing skill across students writing level of MAN Batu students (Doctoral dissertation, Universitas Islam Negeri Maulana Malik Ibrahim). http://etheses.uin-malang.ac.id/id/eprint/36341
Wahyuda, M. I., & Putera, A. A. (2022). The effectiveness of ProWritingAid toward students writing skills in senior high school. English Edu: Journal of English Teaching and Learning, 1(1), 26–34. https://doi.org/10.18860/jetl.v1i1.1624
Widayanti, M. (2011). Enhancing students’ writing skill through task-based approach. Language Circle: Journal of Language and Literature, 6(1), 27-39. https://doi.org/10.15294/LC.V6I1.2044.
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0
Zhang, B. (2013). An analysis of spoken language and written language and how they affect English language learning and teaching. Journal of Language Teaching and Research, 4(4), 834.