The impact of in-text feedback vs. rubric-based feedback on writing performance and self-efficacy of Iranian intermediate EFL learners

Subject Areas : Instructional Design, Planning, and Strategies including ELT

Mahdieh Shafipoor ¹ , Mandana Dalvand ^{2 *}

1 - Department of English Language, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran
2 - Department of English Language, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran

Received: 2024-12-03 Accepted : 2025-02-22 Published : 2025-03-10

Keywords: In-text Feedback, Rubric-based Feedback, Self-efficacy, Writing Performance,

Abstract :

The primary aim of this quasi-experimental study was to delve into the contrasting effects of In-text Feedback and Rubric-based Feedback on the writing performance and self-efficacy of Iranian Intermediate English as a Foreign Language (EFL) learners. To do so, 120 intermediate female English language learners between the ages of 12 and 16 were recruited based on non-random convenience sampling method from an English language institute to take Oxford Placement Test (OPT) to ensure a homogenized group of participants. Then, 60 participants whose scores fell within one standard deviation below and above the mean OPT score were recruited. The selected participants were randomly assigned to one of three groups: the in-text group, the rubric-based group, and the control group, with each group consisting of 20 students. Each group also took writing pre-test and completed self-efficacy questionnaire before the treatment. Next, the participants engaged in the assigned writing activities which were similar and received relevant feedback types for ten sessions. After the treatment, all groups took writing post-test and completed the self-efficacy questionnaire again. The results revealed that both in-text feedback and rubric-based feedback had a statistically significant effect on the writing performance of the learners. Additionally, there was a significant difference in the effect of in-text feedback versus rubric-based feedback on writing performance. In terms of self-efficacy, both types of feedback showed a statistically significant effect. However, there was no significant difference between the effects of in-text feedback and rubric-based feedback on self-efficacy. The findings of the study carry important implications for various stakeholders involved in English as a Foreign Language (EFL) learning, including learners, teachers, language centers, and teacher trainers.

References:

Alfaki, I. (2015). University students' English writing problems: Diagnosis and remedy. International Journal of English Language Teaching, 3(2), 40-52. https://doi.org/10.5296/ijelt.v3i2.7947
Al-Mukdad, S. (2019). Investigating English academic writing problems encountered by Arab international university students. Theory and Practice in Language Studies, 9(3), 300-306. https://doi.org/10.17507/tpls.0903.02
Alnasser, S. M., & Alyousef, H. S. (2015). Improving the effectiveness of the peer feedback technique: The impact of focusing EFL student-writers on macro level features. International Journal of English Language Education, 3(1), 92-112. https://doi.org/10.5296/ijele.v3i1.6764
Al-Saadi, Z. (2020). Gender differences in writing: The mediating effect of language proficiency and writing fluency in text quality. Cogent Education, 7(1), 177-182. https://doi.org/10.1080/2331186X.2020.1727286
Annisa, A., & Gusnawaty, G. (2024). Feedback and revisions in developing writing skills in second language learners: A systematic review. Kajian Pendidikan, Seni, Budaya, Sosial dan Lingkungan, 1(1), 1-8. https://doi.org/10.58881/kpsbsl.v1i1.7
Bailey, A. L., & Huang, B. H. (2011). Do current English language development/proficiency standards reflect the English needed for success in school? Language Testing, 28(3), 343-365. https://doi.org/10.1177/0265532210384903
Bandura, A. (1986). Social foundations of thought and action: A social cognitive view. Prentice Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.
Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88(1), 87-99. https://doi.org/10.1037/0021-9010.88.1.87
Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1-26. https://doi.org/10.1146/annurev.psych.52.1.1
Brooks, C., Burton, R., van der Kleij, F., Ablaza, C., Carroll, A., Hattie, J., & Neill, S. (2021). Teachers activating learners: The effects of a student-centered feedback approach on writing achievement. Teaching and Teacher Education, 105(1), 103-110. https://doi.org/10.1016/j.tate.2021.103343
Brookhart, S. M. (2003). Enhancing classroom assessment through feedback. In S. M. Brookhart (ED.), Assessment, grading, and record keeping in the classroom (pp. 73-98). Eye on Education.
Brown, D. (2007). Principles of language learning and teaching. Pearson Education.
Bürgermeister, A., Glogger-Frey, I., & Saalbach, H. (2021). Supporting peer feedback on learning strategies: Effects on self-efficacy and feedback quality. Psychology Learning & Teaching, 20(3), 383-404. https://doi.org/10.1177/14757257211016604
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233. https://doi.org/10.1080/03075070600680650
Cheng, X., & Zhang, L. J. (2021). Teacher written feedback on English as a foreign language learners’ writing: Examining native and non-native English-speaking teachers’ practices in feedback provision. Frontiers in Psychology, 12(1), 629-633. https://doi.org/10.3389/fpsyg.2021.619964
Cole, J., & Feng, J. (2015). Effective strategies for improving writing skills of elementary English language learners. In Chinese American Educational Research and Development Association Annual Conference (pp. 1-25). Retrieved from http://files.eric.ed.gov/fulltext/ED556123.pdf
Derham, C., Balloo, K., & Winstone, N. (2021). The Focus, function, and framing of feedback information: Linguistic and content analysis of in-text feedback comments. Evaluation in Higher Education, 1(1), 1-14. https://doi.org/10.1080/02602938.2021.1969335
Dirkx, K., Joosten-ten Brinke, D., Arts, J., & van Diggelen, M. (2021). In-text and rubric-referenced feedback: Differences in focus, level, and function. Active Learning in Higher Education, 22(3), 189-201. https://doi.org/10.1177/1469787419855208
Eryilmaz, A., & Yesilyurt, Y. E. (2020). Foreign language writing as a developmental process (foundation, expansion, development, and completion): The FEDCom model. International Journal of Contemporary Educational Research, 7(2), 307-334. https://doi.org/10.33200/ijcer.768768
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland, & F. Hyland, Feedback in Second Language Writing: Contexts and Issues (pp. 81-104). Cambridge University Press.
Graham, S., & Harris, K. (2005). Writing better: Effective strategies for teaching students with learning difficulties. Brookes Publishing Company.
Göçer, A., & Şenel, M. (2017). The effectiveness of different types of feedback on Turkish learners' written accuracy. Language Teaching Research, 21(6), 707-728. https://doi.org/10.33200/ijcer.768768
Hasan, A. A. A. (2022). Effect of rubric-based feedback on the writing skills of high school graders. Journal of Innovation in Educational and Cultural Research, 3(1), 49-58. https://doi.org/10.46843/jiecr.v3i1.52
Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: A meta-analysis. Assessment & Evaluation in Higher Education, 44(6), 863-880. https://doi.org/10.1080/02602938.2018.1521179
Hyland, K. (2003). Second Language Writing. Cambridge University Press.
Jerusalem, M., & Schwarzer, R. (1992). Self-efficacy as a resource factor in stress appraisal processes. In R. Schwarzer (Ed.), Self-efficacy: Thought control of action (pp. 195-213). Routledge.
Lane, J., Lane, A. M., & Kyprianou, A. (2004). Self-efficacy, self-esteem and their impact on academic performance. Social Behavior and Personality: An International Journal, 32(3), 247-256. https://doi.org/10.2224/sbp.2004.32.3.247
Lv, X., Ren, W. & Xie, Y. (2022). The effects of online feedback on ESL/EFL Writing: A meta-analysis. Asia-Pacific Edu Res 30, 643–653. https://doi.org/10.1007/s40299-021-00594-6
Mackey, A., & Gass, S. M. (2015). Second Language Research: Methodology and Design. Routledge.
Mahmoudi, F., & Bugra, C. (2020). The effects of using rubrics and face to face feedback in teaching writing skill in higher education. International Online Journal of Education and Teaching, 7(1), 150-158. https://doi.org/10.15345/iojet.671
Mallahi, O., & Saadat, M. (2020). Effects of feedback on Iranian EFL learners' writing development: group dynamic assessment vs. formative assessment. Iranian Evolutionary Educational Psychology Journal, 2(4), 258-277. https://doi.org/10.52547/ieepj.2.4.258
Mao, Z., Lee, I., & Li, S. (2024). Written corrective feedback in second language writing: A synthesis of naturalistic classroom studies. Language Teaching, 1-29. https://doi.org/10.1017/s0261444823000393
Noroozi, M., & Siyyari, M. (2019). Meaning-focused output and meaning-focused input: The case of passive and active vocabulary learning. Cogent Arts & Humanities, 6(1), 165-172. https://doi.org/10.1080/23311983.2019.1608712
Pablo, J. C., & Lasaten, R. C. (2018). Writing difficulties and quality of academic essays of senior high school students. Asia Pacific Journal of Multidisciplinary Research, 6(4), 46-57. https://www.researchgate.net/publication/328331613_Writing_Difficulties_and_Quality_of_Academic_Essays_of_Senior_High_School_Students
Paltridge, B. (2004). The role of feedback: An analysis of written teacher feedback to ESL student writing. Hong Kong Journal of Applied Linguistics, 8(2), 83-104. https://doi.org/10.4324/9781410611505-9
Pajares, F., & Valiante, G. (1999). Grade level and gender differences in the writing self-beliefs of middle school students. Contemporary Educational Psychology, 24(4), 390-405. https://doi.org/10.1006/ceps.1999.1018
Pajares, F., & Valiante, G. (2001). Grade level and gender differences in the writing self-beliefs of middle school students. Contemporary Educational Psychology, 26(2), 146-166. https://doi.org/10.1006/ceps.2000.1048
Price, F., & Kadi-Hanifi, K. (2011). E-motivation! The role of popular technology in student motivation and retention. Research in Post-Compulsory Education, 16(2), 173-187. https://doi.org/10.1080/13596748.2011.571720
Rahman, M. N. (2017). Incorporating different forms of feedback in teaching writing: An insight into a real classroom. BELTA Journal, 1(1), 64-82. https://doi.org/10.36832/beltaj.2017.0101.04
Rahimi, M. (2021). A comparative study of the impact of focused vs. comprehensive corrective feedback and revision on ESL learners’ writing accuracy and quality. Language Teaching Research, 25(5), 687-710. https://doi.org/10.1177/1362168819879182
Rao, P. S. (2019). The role of English as a global language. Research Journal of English, 4(1), 65-79. https://doi.org/10.1002/9781118784235.eelt0311
Reichelt, M. (1999). Recurrent error patterns in the writing of university students. English for Specific Purposes, 18(4), 313-326. https://doi.org/10.1016/S0889-4906(97)00068-9
Riadil, I. G., & Nur, M. R. (2020). Exploring the impact of extroversion and introversion personality types on EFL learners’ preferences in publishing research papers. International Journal of Education, Language, and Religion, 2(2), 47-53. https://doi.org/10.21462/ijelr.v2i2.57
Richards, J. C., & Schmidt, R. W. (2013). Longman dictionary of language teaching and applied linguistics. Routledge.
Salaxiddinovna, M. G. (2022). Solutions to the problems of teaching writing skills in English in higher education institutions based on foreign manuals. Web of Scientist: International Scientific Research Journal, 3(6), 1782-1785. https://doi.org/10.46763/WoS.2022.03.006
Sarvestani, M. S., & Pishkar, K. (2015). The effect of written corrective feedback on writing accuracy of intermediate learners. Theory and Practice in Language Studies, 5(10), 2046-2052. https://doi.org/10.17507/tpls.0510.10
Shell, D. F., Colvin, C., & Bruning, R. H. (1995). Self-efficacy, attribution, and outcome expectancy mechanisms in reading and writing achievement: Grade-level and achievement-level differences. Journal of Educational Psychology, 87(3), 386-394. https://doi.org/10.1037/0022-0663.87.3.386
Schwartz, R. M., & White, B. T. (2000). The influence of writing mode on revision. In R. M. Schwartz & B. T. White (EDs), Handbook of Writing Research (pp. 409-422). Guilford Press.
Szlachta, B., Polok, K., & Bieńkowska, I. (2023). The importance of feedback in Improving students’ writing skills with the assistance of new technologies. Multidisciplinary Journal of School Education, 12(1), 357-386. https://doi.org/10.35765/mjse.2023.1223.16
Weal, E. (2013). English grammar: Step by step, Books 1 and 2 Gramática del inglés: Paso a paso, Books 1 and 2. CATESOL Journal, 24, 327-345.
Wahyuni, S. (2017). The effect of different feedback on writing quality of college students with different cognitive styles. Dinamika Ilmu, 17(1), 39-58. https://doi.org/10.21093/di.v17i1.649
Xiaoxiao, L., & Yan, L. (2010). A case study of dynamic assessment in EFL process writing. Chinese Journal of Applied Linguistics (Foreign Language Teaching & Research Press), 33(1), 1-20. https://doi.org/10.1515/cjal-2015-0031

Full-Text:



 

 

Curriculum Research		Volume 6, Issue 1 Mar. 10, 2025
The impact of in-text feedback vs. rubric-based feedback on writing performance and self-efficacy of Iranian intermediate EFL learners
Article info	Abstract
Article Type:	The primary aim of this quasi-experimental study was to delve into the contrasting effects of In-text Feedback and Rubric-based Feedback on the writing performance and self-efficacy of Iranian Intermediate English as a Foreign Language (EFL) learners. To do so, 120 intermediate female English language learners between the ages of 12 and 16 were recruited based on non-random convenience sampling method from an English language institute to take Oxford Placement Test (OPT) to ensure a homogenized group of participants. Then, 60 participants whose scores fell within one standard deviation below and above the mean OPT score were recruited. The selected participants were randomly assigned to one of three groups: the in-text group, the rubric-based group, and the control group, with each group consisting of 20 students. Each group also took writing pre-test and completed self-efficacy questionnaire before the treatment. Next, the participants engaged in the assigned writing activities which were similar and received relevant feedback types for ten sessions. After the treatment, all groups took writing post-test and completed the self-efficacy questionnaire again. The results revealed that both in-text feedback and rubric-based feedback had a statistically significant effect on the writing performance of the learners. Additionally, there was a significant difference in the effect of in-text feedback versus rubric-based feedback on writing performance. In terms of self-efficacy, both types of feedback showed a statistically significant effect. However, there was no significant difference between the effects of in-text feedback and rubric-based feedback on self-efficacy. The findings of the study carry important implications for various stakeholders involved in English as a Foreign Language (EFL) learning, including learners, teachers, language centers, and teacher trainers.
Original Research
Authors:
Mandana Dalvand1 Mahdieh Shafipoor2
Article History:
Received: 2024/12/03 Accepted: 2025/02/22 Published: 2025/03/10
	Keywords: In-text Feedback, Rubric-based Feedback, Self-efficacy, Writing Performance

 

---

[1] . Department of English Language, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran. Email: [e-mail]

[2] . Department of English Language, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran (Corresponding Author). Email: [e-mail]

1. Introduction

        English has become an indispensable skill across the globe, and it is recognized as an international language that cuts across various disciplines (Rao, 2019). Writing presents a formidable challenge for learners in terms of language skills (Cole & Feng, 2015). It is widely acknowledged as an intricate and demanding process, often considered the most difficult aspect of language learning (Brown, 2007). Developing and organizing original ideas within the appropriate context adds to the complexity of the issue (Richards & Schmidt, 2013). Consequently, many students exhibit negative attitudes towards writing and lack of motivation to improve their skills, as observed by Price and Kadi-Hanifi (2011).

In addition, writing, among the four language skills, has been unjustly neglected, requiring greater attention from instructors (Riadil & Nur, 2020). Weal (2013) posited that writing poses greater challenges for English learners acquiring it as a second or foreign language. At various educational levels, students are tasked with writing essays and reports as part of their curriculum in foreign and second language learning (Bailey & Huang, 2011).

The complexity of the writing skill is often cited as a reason for learners' struggles, as noted by Alfaki (2015). Salaxiddinovna (2022) reported that both English language learners and their teachers face significant challenges in developing writing skills. Similarly, Eryilmaz and Yesilyurt, (2020) unequivocally described writing as arduous work for any English language learner (ELL). Many students lament their lack of ideas and inability to produce engaging texts, even in their native language (Al-Mukdad, 2019; Pablo & Lasaten, 2018).

Additionally, Xiaoxiao and Yan (2010) highlighted the multifaceted nature of writing in English, encompassing skills such as selecting appropriate topics for specific audiences, generating logical and precise ideas, organizing rich and relevant content, and employing accurate language expressions. These skills demand independent thinking abilities, including classification, evaluation, and synthesis. Hyland (2003) further emphasized the additional elements that contribute to the difficulty of writing and mentioned that mastery in writing involves considering mechanics, content, structure, and style when writing in a second or foreign language. Graham and Harris (2005) described the components of writing performance as content, organization, vocabulary, grammar, and mechanics and emphasized strengthening each of these subskills undoubtedly bolsters writing performance.

Extensive research in this realm clearly indicated that offering students feedback on their writing can play a vital role in improving their writing performance (Annisa & Gusnawaty, 2024; Rahman, 2017; Szlachta et al., 2023). English language learners (ELLs) greatly benefit from writing practice and the invaluable revisions they make upon submitting their written work. Consequently, the final draft of learners' writing can serve as a tangible representation of their growth. Moreover, the nature of the feedback the students receive, exerts a significant impact on their overall writing performance (Mallahi, & Saadat, 2020).

A comprehensive review of the literature has revealed that a considerable body of research has investigated the effectiveness of corrective feedback (CF) in enhancing students' writing performances (Mao et al., 2024; Nagode et al., 2014; Sarvestani & Pishkar, 2015). Most studies in this domain consistently demonstrated the helpfulness and efficacy of the feedback as a tool for improving writing proficiency. A series of recent studies (Brooks et al., 2021; Cheng & Zhang, 2021; Huisman et al., 2019; Mahmoudi & Bugra, 2020) specifically delved into the effectiveness of different types of CF in supporting ELLs in their quest to enhance writing performance. However, despite these efforts, the research community remained engaged in ongoing debates and discussions concerning the interpretation of research findings and the overall benefits of CF.

Moreover, self-efficacy emerges as a significant motivational factor examined by educational psychologists (Lane et al., 2004; Pajares & Valiante, 1999; Shell et al., 1995). Writing self-efficacy refers to individuals' appraisal of their composition, grammar, usage, and technical abilities in successfully completing writing tasks (Pajares & Valiante, 2001). According to Bandura (1986), self-efficacy pertains to learners' confidence in their capacity to succeed, learn new material, and accomplish tasks to the required standard. He also added students with high self-efficacy possess problem-solving strategies that have proven effective in the past, attributing success to their efforts and acknowledging mistakes as part of the learning process. In contrast, low-self-efficacious learners exert less effort, fearing that any attempt will reveal their incompetence. They also opt for less challenging tasks to minimize errors. 

Following an extensive examination of research literature pertaining to the improvement of writing performance through various types of feedback (Alnasser & Alyousef, 2015; Rahimi, 2021; Wahyuni, 2017), the researchers in this study identified an opportunity to contribute new findings to the existing body of knowledge. Specifically, the researchers aimed to explore the impact of in-text feedback and rubric-based feedback on the writing performance and self-efficacy of Iranian intermediate English as a Foreign Language (EFL) learners—an aspect that had remained unexplored in the previous studies. Hence, this study was conducted and the following research questions were posed:

RQ1: Does in-text feedback have any statistically significant effect on the writing performance of Iranian intermediate EFL learners?

RQ2: Does rubric-based feedback have any statistically significant effect on the writing performance of Iranian intermediate EFL learners?

RQ3: Is there any statistically significant difference between the effects of in-text feedback vs. rubric-based feedback on the writing performance of Iranian intermediate EFL learners?

RQ4: Does in-text feedback have any statistically significant effect on the self-efficacy of Iranian intermediate EFL learners?

RQ5: Does rubric-based feedback have any statistically significant effect on the self-efficacy of Iranian intermediate EFL learners?

RQ6: Is there any statistically significant difference between the effects of in-text feedback vs. rubric-based feedback on the self-efficacy of Iranian intermediate EFL learners?

 

2. Review of the Related Literature

        The realm of writing assignments presented a formidable challenge for English students, and it was imperative to provide them with substantial support to enhance their skills. The demands of their future professions necessitated proficiency and precision in writing. Over the past five decades, a variety of pedagogical approaches to teaching L1/L2 writing have emerged, each representing a distinct perspective on the nature of writing, with the aim of helping students enhance their writing skills (Annisa & Gusnawaty, 2024; Rahman, 2017; Szlachta et al., 2023). These instructional methods reflect the significant advancements made in L1/L2 writing situations.

In the realm of teaching writing to English as a Second Language (ESL) or English as a Foreign Language (EFL) students, the function of feedback, its significance, and impact have been central concerns in academic research (Paltridge, 2004; Reichelt, 1999). Scholars and researchers have widely recognized the crucial roles the feedback plays in the writing process, as evident from the vast body of research exploring various forms of feedback and their effects on student writing (Annisa & Gusnawaty, 2024; Szlachta et al., 2023).

Receiving feedback allows students to become aware of specific areas that require improvement and refinement in their written texts. As highlighted by Carless (2006), students who receive feedback during the writing process develop a deeper understanding of their progress and gain valuable guidance on how to enhance their work. Moreover, feedback has the potential to influence students' emotions and behaviors regarding their writing, as well as guide their attention toward writing goals. It serves as a bridge between students' current abilities and expected performance, thus assessing their task achievement and effectiveness in fulfilling their writing objectives (Brookhart, 2003; Schwartz & White, 2000).

Also, self-efficacy beliefs play a fundamental role in shaping human agency, influencing how individuals perceive their ability to perform specific tasks (Bandura, 1997; 2001). These beliefs have a significant impact on various cognitive, motivational, affective, and decision-making processes that determine an individual's actions and outcomes (Bandura, 1997; Bandura & Locke, 2003). Self-efficacy beliefs instill a sense of control and determine individuals' belief in their own capabilities, their resilience in the face of challenges, their emotional well-being, and the choices they make during critical moments (Bandura & Locke, 2003).

Several studies have investigated the impact of feedback on EFL learners' writing performance and self-efficacy. A study conducted by Dirkx et al. (2021) aimed to explore how instructors utilized feedback delivered as in-text comments compared to comments that refered to the rubric, and whether these feedback modalities could be used more effectively. The researchers investigated the nature, intensity, and purpose of the feedback in these two modes. The findings revealed that there were nearly five times as many in-text comments as there were additional comments referring to the rubric. Moreover, the in-text comments were found to contain more process- and feed forward-oriented remarks. In a related study, Ferris (2006) examined the effects of written feedback on ESL learners' writing development. The findings showed that comprehensive corrective feedback that addressed both surface-level errors and higher-order concerns contributed to learners' improvement in writing quality. In another study, Carless (2006) investigated the impact of different types of feedback (direct corrective feedback, indirect corrective feedback, and praise) on students' writing performance. The results indicated that both types of corrective feedback were effective in improving students' writing, with direct corrective feedback leading to greater improvements in accuracy.

Similarly, Göçer and Şenel (2017) investigated the effectiveness of different types of feedback (direct correction, indirect correction, and metalinguistic feedback) on Turkish learners' written accuracy. The results indicated that metalinguistic feedback, which focused on explaining the underlying grammatical rules, had a positive impact on learners' accuracy. Furthermore, Derham et al. (2021) emphasized that in-text comments on feedback often focus primarily on the task without providing additional guidance. Also, their study highlighted the importance of considering linguistic characteristics that can promote self-regulation.

In summary, providing feedback is a crucial tool for enhancing the writing performance and self-efficacy of English as a Foreign Language (EFL) learners. According to recent research (Annisa & Gusnawaty, 2024; Rahman, 2017; Szlachta et al., 2023), feedback that fosters student engagement, comprehension, and action is vital in promoting self-regulation and feedback literacy among students. However, more investigation is necessary to determine the most effective feedback modalities for enhancing EFL learners' writing proficiency and self-efficacy. As a result, the researchers in this study identified an opportunity to contribute new findings to the existing body of knowledge and aimed to explore the impact of in-text feedback and rubric-based feedback on the writing performance and self-efficacy of Iranian intermediate English as a Foreign Language (EFL) learners—an aspect that have remained unexplored in previous studies, yet.

 

3. Methodology

3.1. Design

        This study utilized a quasi-experimental design with two experimental groups and a control group, employing a pre-test, post-test design. This approach was deemed appropriate due to the limitations of selecting a large and randomly assigned sample from the population. Additionally, the study required the implementation of two distinct treatments. The use of pre-test and post-test measures enabled the assessment of changes in participants' self-efficacy and writing performance over time.

3.2. Participants

        In this study 120 intermediate female English language learners between the ages of 12 and 16 from an English language institute in Shahr-e-Qods City were recruited non-randomly and following convenience sampling technique. To ensure a homogenized group of participants, the participants were selected based on their scores on the Oxford Placement Test (OPT) and finally 60 participants whose scores fell within one standard deviation below and above the mean OPT scores were selected. The selected participants were randomly assigned to one of the three groups: the in-text group, the rubric-based group, and the control group, each group consisting of 20 students.

3.3. Instruments

        A range of meticulously selected instruments was employed to conduct the investigation comprehensively and gather reliable data. These instruments were carefully chosen to ensure the accuracy and effectiveness of the research methodology. The following instruments were utilized:

3.3.1. Oxford Placement Tests (OPT)

        The Oxford Placement Test (OPT) served as the primary instrument for selecting 60 homogeneous participants. Developed by Oxford University Press and Cambridge ESOL, the OPT is a widely recognized and respected English language examination that offers teachers a reliable and efficient means of assessing students' language proficiency (Hill & Taylor as cited in Noroozi & Siyyari, 2019). With its straightforward administration and quick completion time of approximately 60 minutes, the OPT was an ideal tool for placement tests and screening purposes in this research. As it assessed an individual's overall English proficiency and was considered a reliable measure, as evidenced by a high level of internal consistency (Cronbach's alpha = .91). Furthermore, previous research had established good construct validity for the examination (Wistner et al., 2009 as cited in Al-Saadi, 2020).

3.3.2. Pre-test and Post-test

        To assess the participants' writing performance, all groups were given a writing task centered on the topic of "characteristics of a successful language learner." This topic selection aligned with the study's focus on examining the participants' writing performance. Prior to the treatment sessions, as well as after the intervention, participants were requested to write an essay on this given topic.

To ensure the evaluation of the participants' writing was done reliably, two expert teachers were responsible for rating the essays. The rating process employed a scale specifically developed for assessing writing performance in IELTS Writing Task 2 by the University of Cambridge. This scale has been widely used and validated for evaluating writing proficiency in academic contexts. Furthermore, it was essential to establish the inter-rater reliability of the two raters.

3.3.3. Self-Efficacy Questionnaire

To assess the participants' self-efficacy, a self-efficacy questionnaire was utilized (Appendix A). This questionnaire incorporated the General Self-Efficacy Scale developed by Jerusalem and Schwarzer (1992). This scale served as a psychometric tool designed to measure individuals' optimism and their perceived ability to handle various challenging situations in life. It consisted of 10 Likert-scale items. Participants read each statement and indicated their level of agreement or disagreement, choosing from options such as "strongly disagree," "moderately disagree," "moderately agree," or "strongly agree." This response format allowed for a nuanced assessment of participants' self-efficacy beliefs.

The reliability analysis of the questionnaire, indicated by Cronbach's alpha, was 0.69 (Jerusalem & Schwarzer, 1992). This value reflected the internal consistency of the scale, indicating the extent to which the items in the questionnaire reliably measured the same construct.

 

3.4. Procedure

        The study started by selecting participants from a pool of 120 intermediate English language learners. To ensure homogeneity among the participants, they took the Oxford Placement Test (OPT). The participants had a designated time of 60 minutes to complete the test, which consisted of 60 multiple-choice questions covering various aspects of the English language.

After scoring the test using the answer key, 60 students whose scores fell within one standard deviation above or below the mean were selected as participants in the study. This criterion ensured a representative sample of intermediate English language learners. The selected participants were then randomly assigned to one of three groups: two experimental groups and a control group. The inclusion of a control group allowed for the comparison of the experimental treatments' effects (Mackey & Gass, 2015). Subsequently, all participants took the self-efficacy questionnaire. This questionnaire aimed to assess the participants' optimism and their perceived ability to handle demanding situations.

As a pre-test, the participants were also given 20 minutes to prepare an essay on the topic of "characteristics of a good language student." The written assignments were evaluated using the writing scoring rubric developed by the University of Cambridge for IELTS Writing Task 2. Two teachers, with over ten years of teaching experience, assessed the essays based on the criteria outlined in the rubric.

In the first experimental group, in-text feedback group, the participants were asked to complete the designated writing assignments and the teacher applied text-processing program (Microsoft Word) to provide in-text feedback on digital text. This program technically offers two different options for in-text feedback, namely, comments (annotations) and track changes. In this study, text-progressing program were used and they provided the possibility to add comments next to the text. These comments were used to place a correction next to the text.

In the second experimental group, known as the rubric-based group, the participants were asked to complete the designated writing assignments similar to the first group. To control for the potential intervening effect of the error correction tool in Microsoft Word, participants in this group were required to write their essays in Microsoft Word and submit them electronically to their teacher. The teacher then provided feedback on two specific aspects of each essay—coherence and cohesion, as well as task achievement—using the corresponding rubrics. The participants were given time to review the provided feedback before working on their next assignment.

In the control group, the participants also completed writing assignments similar to those in the first and second groups (See Appendix B). However, in this group, the teacher collected hard copies of their written work, and gave them some general feedback.

After ten sessions of the treatments, all the participants took a post-test, which mirrored the pre-test. Additionally, they completed the self-efficacy questionnaire once again to gauge any potential changes in their self-efficacy beliefs. The post-test essays were rated by the same two expert teachers who evaluated the pre-test assignments.

3.5. Data Analysis

        To address the research questions and examine the potential changes in participants' writing performance and self-efficacy between the pre-and post-test, the researchers employed an independent-samples t-test. This statistical test assessed whether there was a statistically significant difference in the means of the two groups, allowing for a comparison of the participants' performance and self-efficacy scores before and after the treatment.

To ensure that the data met the assumption of normality, the researchers conducted a one-sample Kolmogorov-Smirnov test. This test examined whether the distribution of the data significantly deviated from a normal distribution.

Additionally, to address the third and sixth research questions, the researchers conducted a one-way ANOVA (analysis of variance) and Turkey's post hoc test. The one-way ANOVA examined the differences between multiple groups, specifically comparing the effects of different feedback modalities on participants' writing performance. The post hoc test further analyzed pairwise comparisons between the groups to identify specific differences. These analyses provided insights into the variations in writing performance among the different treatment groups.

 

4. Results

4.1. The Results of OPT

        In order to select 60 homogenized participants, 120 learners who were at the intermediate level of English language proficiency took part in an Oxford Placement Test (OPT). Table 1 shows the mean and standard deviation of this test.

Table 1. 

The Results of OPT

	N	Minimum	Maximum	Mean	Std. Deviation
OPT	120	30.00	42.00	35.8750	3.79421
Valid N (listwise)	120

 

According to the results of the OPT (M=35.87 and Std.=3.79), 60 English language learners whose scores ranged between one standard deviation above and below the mean were selected and assigned into three groups, i.e., the in-text group, the rubric-based group, and the control group.

4.2 The Result of Inter-Rater Reliability

        Table 4 shows the inter-rater reliability of the pre-test writing, which was rated by two expert raters. It is essential to mention that all 60 participants participated in the writing task in both pre-and post-tests. A Pearson product-moment correlation was run to determine if there was an agreement between the two raters. There was a strong, positive correlation between two raters, which was statistically significant (r = .773, n = 60, p = .001).

Table 2.

Inter-Rater Reliability of Pre-test

		Pre-test (Rater 1)	Pre-test (Rater 2)
Pre-test (Rater 1)	Pearson Correlation	1	.773**
	Sig. (2-tailed)		.000
	N	60	60
Pre-test (Rater 2)	Pearson Correlation	.773**	1
	Sig. (2-tailed)	.000
	N	60	60
**. Correlation is significant at the 0.01 level (2-tailed).

Similar to the inter-rater reliability of the raters in the pre-test, the inter-rater reliability of the raters in the post-test was also computed. Table 3 shows that there was a strong, positive correlation between the two raters, which was statistically significant (r = .742, n = 60, p = .001).

Table 3.

 

Inter-Rater Reliability of Post-test

		Post-test (Rater 1)	Post-test (Rater 2)
Post-test (Rater 1)	Pearson Correlation	1	.742**
	Sig. (2-tailed)		.000
	N	60	60
Post-test (Rater 2)	Pearson Correlation	.742**	1
	Sig. (2-tailed)	.000
	N	60	60
**. Correlation is significant at the 0.01 level (2-tailed).

 

4.3. Normality Test

        In order to check the normality of the data, a one-sample Kolmogorov-Smirnov test was employed. Table 4 shows that the pre-test and post-test results had a normal distribution (p>.05); therefore, parametric tests such could be used.

Table 4. 

Tests of Normality

	Groups	Kolmogorov-Smirnova			Shapiro-Wilk
		Statistic	df	Sig.	Statistic	df	Sig.
Pre-test_Self_Efficacy	Rubric-Based Group	.125	20	.200*	.934	20	.186
	In-Text Group	.125	20	.200*	.934	20	.186
	Control Group	.125	20	.200*	.934	20	.186
Post-test_Self_Efficacy	Rubric-Based Group	.125	20	.200*	.934	20	.186
	In-Text Group	.125	20	.200*	.934	20	.186
	Control Group	.125	20	.200*	.934	20	.186
Pre-test_Writing	Rubric-Based Group	.085	20	.200*	.966	20	.659
	In-Text Group	.085	20	.200*	.966	20	.659
	Control Group	.085	20	.200*	.966	20	.659
Post-test_Writing	Rubric-Based Group	.198	20	.039	.895	20	.034
	In-Text Group	.315	20	.000	.802	20	.001
	Control Group	.213	20	.018	.913	20	.072
*. This is a lower bound of the true significance.
a. Lilliefors Significance Correction

4.4. Reliability of Self-Efficacy Questionnaire

        The reliability of the self-efficacy questionnaire applied to this study was calculated using the Cronbach alpha method. Table 5 shows the reliability coefficient of the questionnaire, which was .76, showing a reasonably acceptable index of reliability coefficient.

Table 5. 

Reliability of Self-Efficacy Questionnaire

Cronbach's Alpha	N of Items
.762	10

 

4.5. Addressing the First Research Question 

        In order to address the first research question of the current study, the researchers employed independent-samples t-test. Table 6 shows the mean scores of the control group (M=4.89) and the in-text group (M=4.72) in the pre-test.

Table 6. 

The Comparison of Groups’ Pre-tests

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Pre-test of Writing	In-Text Group	20	4.9650	.45338	.10138
	Control Group	20	4.8900	.54955	.12288

 

        Table 7 depicts that there was not a statistically significant difference between the pre-test of the control group and the in-text feedback group (P=.389, P>.05).

Table 7. 

Independent Samples Test of Pre-tests

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	.760	.389	.471	38	.640	.07500	.15930	-.24749	.39749
Equal variances not assumed			.471	36.676	.641	.07500	.15930	-.24788	.39788

 

        Table 8 shows the mean scores of the control group (M=4.92) and the in-text feedback group (M=7.15) in the post-test.

Table 8. 

The Comparison of Groups’ Post-test

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Post-test of Writing	In-Text Group	20	7.1500	.23508	.05257
	Control Group	20	4.9200	.71936	.16085

 

        Table 9 reveals that there was a statistically significant difference between the post-test of the control group and the in-text feedback groups (p=.001, P<.05).

Table 9. 

Independent Samples Test of Post-test

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	34.839	.000	13.178	38	.000	2.23000	.16922	1.88742	2.57258
Equal variances not assumed			13.178	23.012	.000	2.23000	.16922	1.87994	2.58006

 

        Therefore, it was confirmed that the in-text feedback had a statistically significant effect on the writing performance of Iranian intermediate EFL learners.

4.6. Addressing the Second Research Question 

        In order to address the second research question, the researchers, an independent-samples t-test was employed. Table 10 depicts the mean scores of the control group (M=4.89) and the rubric-based feedback group (M=5.015) in the pre-test.

Table 10. 

The Comparison of Groups’ Pre-tests

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Pre-test of Writing	Rubric-Based Group	20	5.0150	.44162	.09875
	Control Group	20	4.8900	.54955	.12288

 

        Table 11 shows that there was not a statistically significant difference between the pre-test of control and rubric-based feedback group (P=.199, P>.05).

Table 11. 

Independent Samples Test of Pre-tests

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	1.709	.199	.793	38	.433	.12500	.15764	-.19413	.44413
Equal variances not assumed			.793	36.318	.433	.12500	.15764	-.19462	.44462

 

        Table 12 also reveals the mean scores of the control group (M=4.92) and the rubric-based group (M=7.15) in the post-test.

Table 12. 

The Comparison of Groups’ Post-test

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Post-test of Writing	Rubric-Based Group	20	6.1500	.67473	.15087
	Control Group	20	4.9200	.71936	.16085

 

        Table 13 shows that there was not a statistically significant difference between the post-test of the control group and the rubric-based feedback groups (p=.715, P<.05).

Table 13. 

Independent Samples Test of Post-test

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	.135	.715	5.577	38	.000	1.23000	.22054	.78354	1.67646
Equal variances not assumed			5.577	37.845	.000	1.23000	.22054	.78348	1.67652

 

        As the statistics show, although there was a difference between the mean scores of the rubric-based group and the control group, the difference was not significant. Therefore, it is confirmed that rubric-based feedback did not have a statistically significant effect on the writing performance of Iranian intermediate EFL learners.

4.7. Addressing the Third Research Question

        In order to address the third research question, a one-way ANOVA, and Tukey Post Hoc was deployed. As Table 14 shows, there was a statistically significant difference between the post-test of three groups (P=.001, P<.005).

Table 14. ANOVA: Post-test of Writing
	Sum of Squares	df	Mean Square	F	Sig.
Between Groups	49.905	2	24.953	72.819	.000
Within Groups	19.532	57	.343
Total	69.437	59

 

Table 15 shows the results of the comparison between the three groups’ post-tests. It also shows that the mean scores of the three post-tests were statistically significantly different.

Table 15.

Multiple Comparisons of Post-test

(I) Groups	(J) Groups	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
					Lower Bound	Upper Bound
Rubric-Based Group	In-Text Group	-1.00000*	.18511	.000	-1.4455	-.5545
	Control Group	1.23000*	.18511	.000	.7845	1.6755
In-Text Group	Rubric-Based Group	1.00000*	.18511	.000	.5545	1.4455
	Control Group	2.23000*	.18511	.000	1.7845	2.6755
Control Group	Rubric-Based Group	-1.23000*	.18511	.000	-1.6755	-.7845
	In-Text Group	-2.23000*	.18511	.000	-2.6755	-1.7845
*. The mean difference is significant at the 0.05 level.

 

Table 16 depicts that there was a statistically significant difference between groups in post-tests. A Tukey post hoc test revealed that the in-text feedback group (M=7.15) was statistically significantly higher than the control group (M=4.92) and the rubric-based feedback group (M=6.15).

Table 16.

Turkey Post Hoc Results of Post-tests

Groups	N	Subset for alpha = 0.05
		1	2	3
Control Group	20	4.9200
Rubric-Based Group	20		6.1500
In-Text Group	20			7.1500
Sig.		1.000	1.000	1.000
Means for groups in homogeneous subsets are displayed.
a. Uses Harmonic Mean Sample Size = 20.000.

 

Therefore, it was confirmed that there was a statistically significant difference between the effects of in-text feedback vs. rubric-based feedback on the writing performance of Iranian intermediate EFL learners.

4.8. Addressing the Fourth Research Question 

        The researchers used the independent-samples t-test to deal with the fourth research question of the current investigation. Table 17 displays the mean scores of the in-text feedback group (M=23.85) and the control group (M=24.30) in the pre-test of self-efficacy.

Table 17.

The Comparison of Self-Efficacy of In-Text and Control Groups in Pre-tests

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Pre-test of Self-Efficacy	In-Text Group	20	23.8500	1.98083	.44293
	Control Group	20	24.3000	1.45458	.32525

 

        Table 18 shows that there was not a statistically significant difference between the pre-test of the control and in-text feedback group (P=.141, P>.05).

Table 18.

Independent Samples Test of Pre-tests

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	2.262	.141	-.819	38	.418	-.45000	.54952	-1.56245	.66245
Equal variances not assumed			-.819	34.875	.418	-.45000	.54952	-1.56573	.66573

 

        Table 19 also shows the mean scores of the control group (M=24.25) and the in-text feedback group (M=27.20) in the post-test of self-efficacy questionnaire results.

Table 19.

The Comparison of Self-Efficacy of In-Text and Control Groups in Post-test

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Post-test of Self-Efficacy	In-Text Group	20	27.2000	1.00525	.22478
	Control Group	20	24.2500	2.14905	.48054

 

        Table 20 demonstrates that there was a statistically significant difference between the control group and the in-text feedback groups (p=.001, P<.05) in the post-test of self-efficacy questionnaire results.

Table 20.

Independent Samples Test of Post-test

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	24.002	.000	5.561	38	.000	2.95000	.53052	1.87603	4.02397
Equal variances not assumed			5.561	26.935	.000	2.95000	.53052	1.86135	4.03865

 

        Therefore, it was confirmed that in-text feedback had a statistically significant effect on the self-efficacy of Iranian intermediate EFL learners.

4.9. Addressing the Fifth Research Question 

        The researcher used the independent-samples t-test to address the fifth research question of the current investigation. Table 4.21 shows the mean scores of the rubric-based group (M=23.85) and the control group (M=24.30) in the pre-test of self-efficacy.

 

Table 21.

The Comparison of Self-Efficacy of Rubric-based and Control Groups in Pre-tests

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Pre-test of Self-Efficacy	Rubric-Based Group	20	24.0000	1.777047	.39736
	Control Group	20	24.3000	1.45458	.32525

 

        Table 22 shows that there was not a statistically significant difference between the pre-test of the control and rubric-based group (P=.340, P>.05).

Table 22.

Independent Samples Test of Pre-test

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	.933	.340	.584	38	.563	3.0000	.51350	1.3395	.7395
Equal variances not assumed			.584	36.57	.563	3.0000	.51350	1.3408	.7408

 

        Table 23 also shows the mean scores of the control group (M=24.25) and the rubric-based group (M=27.85) in the post-test of self-efficacy.

Table 23.

The Comparison of Self-Efficacy of Rubric-based and Control Groups in Post-tests

	Groups	N	Mean	Std. Deviation	Std. Error Mean
Post-test of Self-Efficacy	Rubric-Based Group	20	27.8500	1.98083	.44293
	Control Group	20	24.2500	2.14905	.48054

 

        Table 24 reveals that there was not a statistically significant difference between the control group and the rubric-based feedback groups (p=.785, P<.05) in the post-test of self-efficacy questionnaire results.

 

Table 24.

Independent Samples Test of Post-test

Levene's Test for Equality of Variances
			t-test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	.785	.381	5.509	38	.000	3.60000	.65353	2.27699	4.92301
Equal variances not assumed			5.509	37.750	.000	3.60000	.65353	2.27670	4.92330

 

        Therefore, it was confirmed that rubric-based feedback did not have a statistically significant effect on the self-efficacy of Iranian intermediate EFL learners.

4.10. Addressing the Sixth Research Question

        To address the sixth research question, a one-way analysis of variance (ANOVA) was applied and subsequently, a Tukey post hoc analysis was conducted. The obtained results, as presented in Table 25, indicates the presence of a statistically significant difference among the post-test scores of the three groups (P = .001, P < .005).

Table 25.

ANOVA: Post-test of Self-efficacy

			Sum of Squares	df	Mean Square	F	Sig.
Between Groups	(Combined)		147.233	2	73.617	23.119	.000
	Linear Term	Contrast	129.600	1	129.600	40.701	.000
		Deviation	17.633	1	17.633	5.538	.022
Within Groups			181.500	57	3.184
Total			328.733	59

 

        Furthermore, Table 26 illustrates the comparative outcomes of the post-tests for the three groups, clearly demonstrating that the mean scores of the in-text feedback group and the rubric-based feedback group were statistically significant compared to the control group.

 

Table 26.

Multiple Comparisons Post-test

---