TEFL Ph.D. Students' Engagement with Peer Feedback and Automated Feedback on Their Dissertation Proposals
Lotfollah Akbarpour
1
(
English Department, Bushehr Branch, Islamic Azad University, Bushehr, Iran
)
Leila Saberi
2
(
استادیار گروه زبان انگلیسی، واحد مرودشت، دانشگاه آزاد اسلامی، مرودشت، ایران
)
saeed yazdani
3
(
Department of English, Bushehr Branch, Islamic Azad University, Bushehr, Iran
)
کلید واژه: ChatGPT 3.5-generated feedback, peer feedback, academic writing, behavioral engagement,
چکیده مقاله :
This study was conducted to examine the comments provided by ChatGPT 3.5 and doctoral students on doctoral dissertation proposals. Feedback receivers’ behavioral engagement with these two feedback types was also examined. The participants of this study, selected based on convenience sampling, were 28 Teaching English as a Foreign Language PhD students from three provinces who wrote their dissertation proposals in English. The first version and revised versions of their proposals and the provided comments were analyzed to identify the feedback types and the extent they applied the comments. Furthermore, stimulated recall interviews were used to identify the reasons why they did not apply some of the participants. Findings showed that both ChatGPT 3.5 and doctoral students were successful in providing both content-related and form-related comments. These two feedback sources also provided significant numbers of elaborated and justified feedback on dissertation proposals. Feedback receivers applied most of the comments provided by these feedback sources, and the specificity levels of comments affected the incorporation rate. The findings of the thematic analysis of the stimulated recall interview data revealed that the participants did not apply the comments because they were too broad, inaccurate, difficult to apply, and difficult to understand.
چکیده انگلیسی :
This study was conducted to examine the comments provided by ChatGPT 3.5 and doctoral students on doctoral dissertation proposals. Feedback receivers’ behavioral engagement with these two feedback types was also examined. The participants of this study, selected based on convenience sampling, were 28 Teaching English as a Foreign Language PhD students from three provinces who wrote their dissertation proposals in English. The first version and revised versions of their proposals and the provided comments were analyzed to identify the feedback types and the extent they applied the comments. Furthermore, stimulated recall interviews were used to identify the reasons why they did not apply some of the participants. Findings showed that both ChatGPT 3.5 and doctoral students were successful in providing both content-related and form-related comments. These two feedback sources also provided significant numbers of elaborated and justified feedback on dissertation proposals. Feedback receivers applied most of the comments provided by these feedback sources, and the specificity levels of comments affected the incorporation rate. The findings of the thematic analysis of the stimulated recall interview data revealed that the participants did not apply the comments because they were too broad, inaccurate, difficult to apply, and difficult to understand.
Al Qunayeer, H. S. (2020). Supporting postgraduates in research proposals through peer feedback in a Malaysian university. Journal of Further and Higher Education, 44(7), 956-970.
Awidi, I. T. (2024). Comparing expert tutor evaluation of reflective essays with marking by generative artificial intelligence (AI) tool. Computers and Education: Artificial Intelligence, 6, 100226.
Berndt, M., Strijbos, J.-W., & Fischer, F. (2018). Effects of written peer-feedback content and sender’s competence on perceptions, performance, and mindful cognitive processing. European Journal of Psychology of Education, 33, 31-49.
Bitchener, J. (2017). Why some L2 learners fail to benefit from written corrective feedback. In Corrective feedback in second language teaching and learning (pp. 129-140). Routledge.
Bolzer, M., Strijbos, J.-W., & Fischer, F. (2015). Inferring mindful cognitive‐processing of peer‐feedback via eye‐tracking: Role of feedback‐characteristics, fixation‐durations and transitions. Journal of Computer Assisted Learning, 31(5), 422-434.
Burstein, J., Chodorow, M., & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing service. Ai magazine, 25(3), 27-27.
Chen, C. W.-y. (2010). Graduate students’ self-reported perspectives regarding peer feedback and feedback from writing consultants. Asia Pacific Education Review, 11, 151-158.
Cotos, E. (2016). Computer-assisted research writing in the disciplines. In Adaptive educational technologies for literacy instruction (pp. 225-242). Routledge.
Davin, K. J. (2013). Integration of dynamic assessment and instructional conversations to promote development and improve assessment in the language classroom. Language Teaching Research, 17(3), 303-322.
Ellis, R. (2010). Epilogue: A framework for investigating oral and written corrective feedback. Studies in second language acquisition, 32(2), 335-349.
Fan, Y., & Xu, J. (2020). Exploring student engagement with peer feedback on L2 writing. Journal of Second Language Writing, 50, 100775.
Fernando, W. (2020). Moodle quizzes and their usability for formative assessment of academic writing. Assessing Writing, 46, 100485.
Han, Y. (2017). Mediating and being mediated: Learner beliefs and learner engagement with written corrective feedback. System, 69, 133-142.
Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of second language writing, 30, 31-44.
Jacobs, H. L. (1981). Testing ESL composition: A practical approach. English composition program. ERIC.
Knight, S., Shibani, A., Abel, S., Gibson, A., & Ryan, P. (2020). AcaWriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research.
Lachner, A., & Neuburg, C. (2019). Learning by writing explanations: Computer-based feedback about the explanatory cohesion enhances students’ transfer. Instructional Science, 47(1), 19-37.
Law, L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 100174.
McNamara, D. S., Crossley, S. A., & Roscoe, R. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45, 499-515.
Mehrpour, S., Hoomanfard, M. H., & Vazin, E. (2023). Peer Feedback Accuracy in Synchronous and Asynchronous Computer-mediated Conditions in an EFL Context. Iranian Journal of Language Teaching Research, 11(1), 97-116.
Octavio, M. M., Argüello, M. V. G., & Pujolà, J.-T. (2024). ChatGPT as an AI L2 teaching support: A case study of an EFL teacher. Technology in Language Teaching & Learning, 6(1), 1-25.
Pearson, W. S. (2022). Response to written commentary in preparation for high-stakes second language writing assessment. Asian-Pacific Journal of Second and Foreign Language Education, 7(1), 19.
Poehner, M. E., & Wang, Z. (2021). Dynamic assessment and second language development. Language Teaching, 54(4), 472-490.
Rosa, E. M., & Leow, R. P. (2004). Awareness, different learning conditions, and second language development. Applied psycholinguistics, 25(2), 269-292.
Sachs, R., & Polio, C. (2007). Learners' uses of two types of written feedback on a L2 writing revision task. Studies in Second Language Acquisition, 29(1), 67-100.
Saeed, M. A., & Ghazali, K. (2019). Engaging postgraduates in a peer research group at the research proposal stage in a Malaysian university: support and challenges. Teaching in Higher Education, 24(2), 180-196.
Sedikides, C., Luke, M. A., & Hepper, E. G. (2016). Enhancing feedback and improving feedback: Subjective perceptions, psychological consequences, behavioral outcomes. Journal of Applied Social Psychology, 46(12), 687-700.
Sluijsmans, D. M., Brand-Gruwel, S., van Merriënboer, J. J., & Bastiaens, T. J. (2002). The training of peer assessment skills to promote the development of reflection skills in teacher education. Studies in Educational Evaluation, 29(1), 23-42.
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J.,…Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894.
Stevenson, M., & Phakiti, A. (2019). Automated feedback and second language writing. Feedback in second language writing: Contexts and issues, 125-142.
Suzuki, W., Nassaji, H., & Sato, K. (2019). The effects of feedback explicitness and type of target structure on accuracy in revision and new pieces of writing. System, 81, 135-145.
Van der Kleij, F. M., & Lipnevich, A. A. (2021). Student perceptions of assessment feedback: A critical scoping review and call for research. Educational assessment, evaluation and accountability, 33, 345-373.
Vuogan, A., & Li, S. (2023). Examining the effectiveness of peer feedback in second language writing: A meta‐analysis. Tesol Quarterly, 57(4), 1115-1138.
Wang, N., Wang, X., & Su, Y.-S. (2024). Critical analysis of the technological affordances, challenges and future directions of Generative AI in education: a systematic review. Asia Pacific Journal of Education, 44(1), 139-155.
Wu, Y., & Schunn, C. D. (2020). From feedback to revisions: Effects of feedback features and perceptions. Contemporary Educational Psychology, 60, 101826.
Yu, S. (2019). Learning from giving peer feedback on postgraduate theses: Voices from master's students in the Macau EFL context. Assessing Writing, 40, 42-52.
Yu, S., Jiang, L., & Zhou, N. (2020). Investigating what feedback practices contribute to students’ writing motivation and engagement in Chinese EFL context: A large scale study. Assessing Writing, 44, 100451.
Yu, S., & Lee, I. (2016). Peer feedback in second language writing (2005–2014). Language Teaching, 49(4), 461-493.
Zhai, N., & Ma, X. (2023). The effectiveness of automated writing evaluation on writing quality: A meta-analysis. Journal of Educational Computing Research, 61(4), 875-900.
Zhang, Z. (2017). Student engagement with computer-generated feedback: A case study. ELT Journal, 71(3), 317-328.
