Computer-based Feedback through Grammarly: Impact on Iranian Intermediate EFL Learners’ Writing Skills and Attitudes
الموضوعات : نشریه زبان و ترجمه
Mohden Ashrafganjoe
1
(English Department, Maybod Branch, Islamic Azad University, Maybod, Iran)
Mohammad Javad Rezai
2
(English Department, Yazd University, Yazd, Iran)
Seyede Elham Elhambakhsh
3
(English Department, Yazd University, Yazd, Iran)
الکلمات المفتاحية: Computer-based Feedback, Grammarly®, software, L2 Writing Skills, Teacher Feedback, Writing Attitudes,
ملخص المقالة :
Abstract The present study was an attempt to investigate the effects of computer-based feedback through Grammarly® on the L2 writing skills and writing attitudes of Iranian EFL learners. Forty lower intermediate and 40 upper intermediate level male and female learners with the age range of 20 to 26 studying TEFL were chosen through a standard version of OQPT as the homogeneity test. These participants represented the two experimental sub-groups namely, Grammarly® Correction Group (GCG) and the Teacher-Correction Group (TCG). To apply the treatment, the subjects in the GCG were asked to develop eight writing tasks all in 150-190 words during the treatment using the Grammarly® software. However, in TCG, the teacher himself provided feedback and assessed the students’ process of learning. Following intervention sessions, the writing posttest was run to measure the learners’ ability in L2 writing skills. Employing two sets of Two-way analysis of covariance (Two-way ANCOVA) and a multivariate analysis of covariance (MANCOVA), the learners' attitude towards Grammarly ® by proficiency levels was collected and then a thematic analysis was conducted to analyze the interview data. The findings of the present investigation revealed Grammarly® software had a positive and significant effect on improvement of the EFL learners’ writing skills. Moreover, the results showed that the participants had a positive attitude after taking the instruction based on Grammarly® software. The findings have implications for EFL classroom.
Ahmadi, M. A. (2018). The use of technology in English language learning. International Journal of Research in English Education, 3(2), 115-125.
Alipour, F., Fathi, J., & Saeedian, A. (2019). Enhancing vocabulary learning and self-regulation via a mobile application: An investigation of the Memrise App. Journal of Modern Research in English Language Studies 5(1), 27-46.
Amrhein, H.R., & Nssaji, H. (2010). Written corrective feedback: What do students and teachers prefer and why? Canadian Journal of Applied Linguistics, 13(2), 95-127.
Belali, J., & Sadeghi, K. (2019). The role of corrective feedback timing in task engagement and oral performance. Applied Research on English Language, 9(2), 229-252.
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17(1). 34.
Brockbank, A., & McGill, I. (1998). Facilitating reflecting learning in higher education, Buckingham, England: Society for Research into Higher Education & Open University Press.
Chirimbu, S., & Tafazoli, D. (2013). Technology & media: Applications in language classrooms (TEFL, TESL & TESOL). Professional Communication and Translation Studies, 6(1-2), 187-194.
Cheng, G. (2017). The impact of online automated feedback on students' reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18–27.
Creswell, J. W., & Hirose, M. (2019). Mixed methods and survey research in family medicine and community health. Family Medicine and Community Health, 7(2), 1-6.
Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage publications.
Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17.
El Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2), 121-142.
Esfahani, F., Hashemifardnia, A., & Namaziandost, E. (2018). The effect of using WhatsApp on Iranian EFL learners’ vocabulary learning. Journal of Applied Linguistics and Language Research, 5(3), 256-267.
Ellis, R. (2009b). A typology of written corrective feedback. English Language Teaching, 66, 97-107. doi: 10.1093/elt/ccn023.
Ghahari, SH., Piruznejad, M. (2016). Recast and explicit feedback to young language learners: impacts on grammar uptake and willingness to communicate. Issues in Language Teaching (ILT),5(2), 187-209.
Ghafoori, B., Dastgoshadeh, A., Aminpanah, A., & Ziaei, S. (2016). The effect of CALL on Iranian EFL learners’ grammar of writing. International Journal of Language Learning and Applied Linguistics World, 12(3), 14-23.
Ghufron, M. A., & Rosyida, F. (2018). The role of Grammarly in assessing English as a Foreign Language (EFL) writing. Lingua Cultural, 12(4), 395-403.
Gulley, D. S. (2003). Grammarly® software glosses and their effect on L2 text comprehension and vocabulary learning. Language learning & Technology, 13(2), 48-67.
Hoang, G. T. L., & Kunnan, A. J. (2016). Automated essay evaluation for English language learners: A case study of MY Access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. Journal of Second Language Writing, 8, 1-10.
Hazarika, Z. (2017). Exploring the impact of technology in teaching English: TESOL In the context. European Journal of English Language and Literature Studies, 5(10), 19-28.
Jahangard, A., Rahimi, A., Norouzizadeh, M. (2020). Students’ attitudes towards computer-assisted language learning and its effect on their EFL writing. International Journal of Learning and Teaching. 12(3), 14-22.
Jean, G., & Simard, D. (2011) Grammar learning in English and French L2: Students’ and teachers’ beliefs and perceptions. Foreign Language Annals, 44(3), 465-492.
Kang, T. (2021). The effect of using technology in l2 listening: A meta-analysis of studies from 1980 to 2020. In V. Turel (Ed.), Design solutions for adaptive hypermedia listening software (pp. 1-30). IGI Global.
Kia Heirati, J., & Ahmadi Alashti, L. (2015). Attitudes toward using the Internet for language learning: A case of Iranian English teachers and learners. International Journal of Research Studies in Educational Technology, 4(1), 63-78.
Lenkaitis, C. A. (2020). Valuing technology in the L2 classroom: Student and teacher perceptions, preferences, and digital identity. In M. Kalantzis & B. Cope (Eds.), Handbook of research on digital learning (pp. 173-189). IGI Global.
Li, S. (2018). What is the ideal time to provide corrective feedback? replication of li, Zhu & Ellis (2016) and Arroyo & Vilmaz (2018). Language Teaching, 53(1), 1-13.
Marzban, A., Mojtahedzadeh, S.S. (2014). Collaborative negotiated feedback versus teacher-written feedback: impact on Iranian intermediate EFL learners‟ writing. Theory and Practice in Language Studies, 4(2), 293-302.
Merriam-Webster Dictionary (2016). Merriam-Webster, Inc.: New York, NY, USA.
Motallebzadeh, K., & Nematizadeh, S. (2011). Does gender play a role in the assessment of oral proficiency? English Language Teaching, 4(4), 165-172.
O’Neill, R., & Russell, A. M. T. (2019). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1), 42-56.
Parra G., L., & Calero S., X. (2019). Automated writing evaluation tools in the improvement of writing skills. International Journal of Instruction, 12(2), 209-226.
Patel, C. (2013). Use of multimedia technology in teaching and learning communication skill: An analysis. International Journal of Advancements in Research & Technology, 2(7), 116-123.
Paulus, M., T. (1999). The effect of peer and teacher feedback on student writing. Journal of Second Language Writing, 8(3), 256-289.
Perelman, L. (2017). Automated essay scoring and NAPLAN: A summary report. Retrieved from https://www.nswtf.org.au/files/automated_essay_scoring_and_naplan.pdf.
Rafanello, D. (2008). Writing well: It’s all about attitude. Professional Writing, 2(3), 58-60.
Ruetten, M. K. (2003). Developing composition skills: Rhetoric and grammar. Boston, Heinle.
Qassemzadeh, A., & Soleimani, H. (2016). The impact of feedback provision by Grammarly software and teachers on learning passive structures by Iranian EFL learners. Theory and Practice in Language Studies, 6(9), 1884-1894.
Sahragard, R., & Sadri, M. (2009). On the relationship between multiple intelligences, vocabulary learning knowledge, and vocabulary learning strategies among the Iranian EFL learners. The Iranian EFL Journal Quarterly, 3(82), p.18.
Schachter J, (1991). Corrective feedback in historical perspective. Interlanguage studies bulletin, 7(2), 89-102.
Schraudner, M. (2013). The online teacher’s assistant: Using automated correction programs to supplement learning and lesson planning. Taiwan: Asia University.
Sheen, Y. (2007). The effect of focused written corrective feedback and language aptitude on ESL learners’ acquisition of articles. TESOL Quarterly, 41, 255–283.
Shokrpour, N., Fallahzadeh, M.H. (2007). A Survey of the Students and Interns EFL Writing Problems in Shiraz University of Medical Sciences. The Asian EFL Journal, 9(1), 147-163.
Sormunen, M. (2014). Opinions about EFL grammar learning and teaching: A study of Finnish upper-secondary-school students. Master’s Thesis, University of Jyväskylä.
Thornbury, S. (1999). How to teach grammar. Harlow, Pearson Education.
Zhang, Z., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36(1), 90-102.
Computer-based Feedback through Grammarly: Impact on Iranian Intermediate EFL Learners’ Writing Skills and Attitudes
Mohsen Ashrafganjoe1, Mohammad Javad Rezai2*, Seyede Elham Elhambakhsh3
1PhD Candidate, Department of English, Maybod Branch, Islamic Azad University, Maybod, Iran
2Associate professor of Applied Linguistics, Yazd University, Yazd, Iran
3Assistant Professor of TEFL, English Department, Yazd University, Yazd, Iran
2023/02/11 2023/03/26
Abstract
The present study was an attempt to investigate the effects of computer-based feedback through Grammarly® on the L2 writing skills and writing attitudes of Iranian EFL learners. Forty lower intermediate and 40 upper intermediate level male and female learners with the age range of 20 to 26 studying TEFL were chosen through a standard version of OQPT as the homogeneity test. These participants represented the two experimental sub-groups namely, Grammarly® Correction Group (GCG) and the Teacher-Correction Group (TCG). To apply the treatment, the subjects in the GCG were asked to develop eight writing tasks all in 150-190 words during the treatment using the Grammarly® software. However, in TCG, the teacher himself provided feedback and assessed the students’ process of learning. Following intervention sessions, the writing posttest was run to measure the learners’ ability in L2 writing skills. Employing two sets of Two-way analysis of covariance (Two-way ANCOVA) and multivariate analysis of covariance (MANCOVA), the learners' attitude towards Grammarly ® by proficiency levels was collected and then a thematic analysis was conducted to analyze the interview data. The findings of the present investigation revealed Grammarly® software had a positive and significant effect on the improvement of the EFL learners’ writing skills. Moreover, the results showed that the participants had a positive attitude after taking the instruction based on Grammarly® software. The findings have implications for classrooms.
Keywords: Computer-based Feedback, Grammarly® Software, L2 Writing Skills, Teacher Feedback, Writing Attitudes
INTRODUCTION
English language teaching has been changed with the emergence of technology. Technology makes teaching very amusing and attractive for language learners and offers a lot of choices for them. In traditional methods that have been used in some classrooms, the teachers write everything on the blackboard. The teacher is the authority source in the class and explains everything and the students just listen to their teacher. Regarding the improvement of technology, these methods need to be changed. Using technology helps language learners to grasp the structure of vocabulary and language more efficiently. When the teachers use movies, print text, and on-line communication in their classrooms, it can increase the students’ linguistic knowledge (Patel, 2013).
Hazarika (2017) emphasized that rapid extension of computer-based packages like software, applications, and different internet-based online systems within the English language learning field seem to have a positive impact on promoting learners’ ability to communicate, creating greater satisfaction and autonomy among learners, and also making the educational process more straightforward and accessible. To expand the international acquisition, English courses as a Foreign Language (EFL) are being currently designed and enlarged in many countries to improve the four language main skills among English learners.
Among the four language skills, writing as an integral part of any language is associated with the complicated structures related to word spelling, pronunciation, vocabulary, and grammatical structure. Language teachers strongly believe that it is one of the most difficult language-learning skills for many EFL learners. Indeed, the different technology-based pedagogical applications that are used in writing skills are derived from various problems in traditional teaching and learning methods.
The problems which are related to the learners’ underachievement in grammatical and writing skills are twofold in Iran. The first problem, according to Sahragard and Sadri (2009), is that “students cannot reach the proper level to use the acquired knowledge, and the second problem, is that new writing-based software interventions are rarely used in learning methods as a solution to receive better feedback” (p.18). Iranian EFL learners have problems both in their language learning and especially in their writing skills. Learning writing skills is so crucial for EFL learners (Shokrpour & Fallahzade, 2007). Moreover, due to the number of learners in each classroom and the lack of enough available time, teachers cannot correct all students ‘errors and provide appropriate feedback for all learners (Qassemzadeh & Soleimani, 2016). Thus, all students’ mistakes cannot be corrected by the teacher because of the time assigned to every student’s correction is not enough and many of the students’ mistakes and writing problems remain uncommented. It might be the reason why language learners have positive attitudes towards the use of technologies (Bitchener, 2008; Jahangard, Rahimi & Norouzizadeh, 2020; Marzban & Mojtahedzadeh, 2014). Grammarly® is one of the new online software with different CFs, specifically designed to assist English learners in improving writing capabilities.
Although the impacts of Grammarly® on improving writing skills has been investigated in different studies (Ahmadi, 2018; Bikowski, 2018; Chirimbu & Tafazoli, 2013; Ghafoori, Dastgoshadeh, Aminpanah & Ziaei, 2016; Parra & Calero, 2019; Qassemzadeh & Soleimani, 2016), no one to the best of the researcher’s knowledge has probed its applications in Iran. More to the point, they have only limited their studies to one or two grammar-checking features. Thus, the current study that makes it innovative compared to the previous studies is that it employs Grammarly® to analyze the effect of the immediate corrective feedback on improving EFL learners’ four different writing skills (task response, cohesion, coherence, grammatical accuracy and lexical resources). However, it is worth mentioning that although these grammatical features may not be very important at the average language learning level, if used incorrectly or inappropriately, they may lead to serious distortion and breakdown in communication.
LITERATURE REVIEW
Technology-Enhanced Language Learning (TELL)
Technology-Enhanced Language Learning (TELL) deals with the impact of technology on teaching and learning procedures. Technologies that are used for language learning and instruction are developing so fast that there is a need to keep track of their applications and changes. Esfehani, Hashemifardnia and Namaziandost (2018) attempted to evaluate the possible impacts of employing WhatsApp on vocabulary learning among 50 Iranian female EFL learners in Adiban English language institute, Khuzestan, Iran. The results showed that using WhatsApp significantly improved Iranian EFL learners’ vocabulary learning. This study concluded that chatting and learning via WhatsApp is beneficial to EFL learners to learn the English language. Thus, from the results of these studies, it can be concluded that the true combination of technological applications and teaching methodology is very important to attract learners’ attention to English language learning.
Language Learners’ Attitudes toward Technology
Language learners’ attitudes towards computer-based technologies as teaching-aided tools in English classes may have a great impact on the success or failure of the teaching and learning processes (Heirati & Ahmadi, 2015). Technology-based instruction has paved the way for students to use technology in comparison to other traditional methods that are used in language classes due to its effect on learners' educational lives. Moreover, writing-aided software can be seen as an alternative to traditional role of teachers in language classes but as an effective part of the pedagogical course. Strictly speaking, this is the reason for why most of language learners have a positive attitude toward using computer-based technologies within their classes (Jahangard, Rahimi & Norouzizadeh, 2020).
Grammarly® Software
Grammarly® is one of the new online software with different corrective feedbacks, specifically designed to assist English learners in improving writing capabilities. One of the computer software (automatic internet-based software) that can be implemented in EFL writing class is ‘Grammarly®’. According to Schraudner (2013), Grammarly® was designed with the ability to immediately correct English learners writing errors such as correcting the word spelling, correcting grammatical errors, punctuation correction, checking lexical appropriateness, sentence cohesion, and some other useful writing algorithms. One study conducted by Ghufron and Rosyida (2018) found this software to be more effective to reduce errors in terms of vocabulary usages (diction), language use (grammar), and mechanics of writing (spelling and punctuation). Apparently, this application has had its advantages in promoting not only the writing skill of EFL learners but also may have benefits for learning the language with appropriate teacher’s monitoring and guidance.
Corrective Feedback (CF)
The issue of feedback timing has received much attention thanks to the rapid extension of computerized tools in pedagogical environments (Belali & Sadeghi, 2019). CF has been divided into six main categories; explicit correction, recast, metalinguistic feedback, elicitation, repetition, and clarification request. Generally, explicit correction and metalinguistic are classified as explicit whereas recast and clarification requests are considered as the implicit types of CF (Ghahari & Piruznejad, 2016; Sheen, 2007). The explicit types are classified in the oral CF category while the implicit feedbacks belong to the written CF category.
In writing skills, CF has been described as a teacher's move to attract the learners’ attention to be aware of the grammatical accuracy of the utterance or written text (Ellis, 2009b). According to El Ebyary and Windeatt (2010) and O’Neil & Russell (2019), students used few preplanning strategies and perceptions to feedback improved after using criterion as an educational software. They recognized that teacher intervention is critical for motivation and program success, and the integration of AWE into conventional writing instruction will be useful. It was also concluded that the learners who used Grammarly® showed a positive attitude toward Grammarly® and enjoyed AWE meaningfully more than the feedback provided by their teachers. The amount of time that was allocated to the feedback was satisfactory for the students. Moreover, the learners in the experimental group believed that the feedback provided for them by Grammarly® was very useful. The correction has been postponed by the teacher in order to gather the written work and answer them in written CF (Ellis, 2009b). Moreover, the learner’ erroneous sentences have been modified or the correction will be delayed by teachers in oral CF. However, this decision should be based on the teachers' regards as to which type of feedback can bring the most effective outcome. The present study tries to describe the effect of metalinguistic feedback provided by Grammarly® and teacher feedback on L2 learners’ writing errors. A description of each feedback will be presented in the next sections in detail.
Teacher Corrective Feedback
The role of teacher feedback in L2 writing is so crucial. Teacher feedback, whether oral or written, helps language learners to discover their weaknesses and their strengths in a learning situation. It focuses on grammar, vocabulary, pronunciation, organization and content as aspects of discourse to form language. Teacher feedback has its own merits; on the other hand, learners can discover their own weaknesses and their strengths points in learning a language and teachers can obtain enough beneficial information about the students’ learning situation which in turn will be beneficial to the teaching and learning (Zhong, Yan, & Zou, 2019). In L2 writing, instructors, researchers, and writing scholars believe that teacher feedback is so important to improve learners’ writing (Tang & Liu, 2018). Regarding rhetorical and content aspects of writing, providing such feedback range from error correction to commentary feedback (Goldstein, 2004).
Metalinguistic Corrective Feedback
Metalinguistic corrective feedback can be defined as providing questions, comments, and information by teacher on the error produced by the learner (Abdollahzadeh, 2016). When language learners have enough implicit knowledge about a specific grammatical feature metalinguistic CF can facilitate the process of language learning. According to Chandler (2003), metalinguistic CF can help learners to understand how to correct the form directly in case they cannot do it through explicit CF. It can be so important for the learners who have limited L2 proficiency to enhance their oral abilities. According to Ferris and Roberts (2001) and Chandler (2003), language learners can improve their abilities to correct the errors through metalinguistic CF. Thus, they have access to the corrections when revising, so they only need to consider the corrections. Truscott and Hsu (2008) believed that grammatical accuracy among learners did not improve through metalinguistic CF. Sheen (2007) reported that metalinguistic CF had a positive impact on language learners’ aptitude. It should be noted that metalinguistic CF included not only metalinguistic information about nature of error but also it included the correct form of the error made by student (Azizi Khah & Farahian, 2016).
All these studies found that technology as a writing-aided CF was more beneficial and practical in controlling students learning as compared to teachers’ feedback in the traditional way. Thus, the study aims to conduct an analysis to evaluate the empirical use of grammar-checking software called Grammarly® on the writing skills among EFL learners. Hence, the present inquiry addressed the following research questions:
Q1. Which group of the EFL learners (lower intermediate/ upper intermediate) are more benefited from the immediate corrective feedback provided by Grammarly® software?
Q2. Is there a statistically significant difference between Grammarly® corrective feedback and teacher corrective feedback on lower and upper intermediate EFL Learners writing?
Q3. Is there a statistically significant difference between Grammarly® corrective feedback and teacher corrective feedback on lower and upper-intermediate EFL Learners L2 writing skills (Task Response, Cohesion and Coherence, Grammatical Accuracy and Lexical Resources)?
Q4. What are the Iranian EFL learners’ attitudes about their L2 writing skills development after utilizing the Grammarly® software to provide the immediate corrective feedback?
METHOD
Design and Context of the Study
The current study employed a sequential explanatory mixed methods design in which both quantitative and qualitative data collection and data analysis methods were taken into consideration. This study enjoyed a quasi-experimental design in its quantitative phase, the effects of two interventions (Teacher Correction Feedback (TCF) and Grammarly® software supported immediate Correction Feedback) as independent variables on EFL learners’ writing and writing skills (grammatical range & accuracy, lexical resource, task response, and cohesion & coherence), as dependent variables were taken into consideration. In the qualitative phase, the study enjoyed a non- experimental design in which the views of the study participants and their perceptions toward the method used in the experimental groups were elicited through a semi-structured interview.
Participants
The participants in the current study were 40 lower and 40 upper-intermediate EFL learners, and all were of typical university age, 20 and 26 years old, and participated on a voluntary basis and then each group was further divided into a control and an experimental group and each group included 20 participants. These participants represented the two sub-groups namely, Grammarly® Correction Group (GCG) who enjoyed using the Grammarly® software to receive feedback and the Teacher-Correction Group (TCG) who received the teacher feedback.
Materials and Instruments
In order to collect the required data before and after the treatment, the researcher utilized a number of instruments: a standard version of Oxford Quick Placement Test, a task of writing as the pretest, another task of writing as the posttest, writing rubric, writing attitude questionnaire (Rafanello, 2008) and a semi-structured interview were the main tools used in the current study. A description of each instrument is presented below.
Oxford Quick Placement Tests (OQPT)
For the current study, OQPT was used to check the homogeneity of students. In order to have homogenous groups of participants, those learners whose scores fall between 30 to 39 will be selected as the lower intermediate participants (B1 level). The participants with the scores of 40 to 47 will be labeled as upper intermediate learners (B2 level). The descriptive statistics for the OQPT were presented in Table 1. The KR-21 reliability of the test was .91. The test has been also reported to enjoy high construct validity (Motallebzadeh & Nematizadeh, 2011). The descriptive statistics for the OQPT are presented in Table 1.
Table 1
Descriptive Statistics and KR-21 Reliability of Oxford Quick Placement Test
| N | Mean | Std. Deviation | Variance |
OQPT | 120 | 36.73 | 11.520 | 132.718 |
KR-21 | .91 |
|
|
|
Writing Pretest and Posttest
A pretest and posttest of writing selected through Test of Written English (TWE) available at Educational Testing Service (ETS) website (https://www.ets.org) were the second instruments in this study. The students were selected after the pre-test of language proficiency, the students were selected following that the current task was given to them; moreover, the writing posttest was run to measure the learners’ ability in writing skills following the experiment. The participants’ writings were scored by employing writing rubric and scale presented by Cambridge University and the British Council. including the four sub-categories of task response, grammatical range and accuracy, cohesion and coherence, and lexical resources. To estimate the reliability of the tests, an inter-rater reliability was used.
Writing Attitude Questionnaire
A writing attitude questionnaire including 20 items measuring the writing attitude of EFL learners was administered in the two experimental sub-groups both before and after the intervention. Table 2 displays the Cronbach’s alpha reliability indices for the pretest and posttest of attitude questionnaires. The reliability indices for the pretest and posttest of attitude questionnaires were .964 and .975, respectively. Likewise, the external validity of the scale was established through the significant correlation between students’ self-reported attitude scores with observed measures of their writing attitude (Wright et al., 2019). In addition, the questionnaire was proved to have high construct validity as reported (Rafanello, 2008) as all the items could fit well in the model ensued from the confirmatory factor analysis.
Table 2
Cronbach’s Alpha Reliability Statistics of Attitude Questionnaire
| Cronbach's Alpha | N of Items |
Pretest | .964 | 20 |
Posttest | .975 | 20 |
Semi-Structured Interview
Coupled with the quantitative data, a semi-structured interview was designed for the aim of qualitative data collection. The items of the interview guide were reviewed by three experts who were TEFL PhD holders from Islamic Azad University. Hence, the instrument was confirmed through expert judgement validity.
Data Collection Procedures
Pretreatment Phase
The first phase of the study started in February 2020. The selected learners in two groups- Grammarly® Correction Group (GCG) and teacher-correction group (TCG)- received a writing task as the pretest of writing to see how well they were able to write in English before they experienced the treatment. The writing command and skills of all the participants were measured by the pretest in which they had to complete a writing task of 150-190 words. This task was given to the participants as an assignment and the participants had to finish it and hand in their writings on the due date. All the sub-groups of the study took similar instructions for their ordinary writing courses. The experimental sub-group learners received the treatment thorough Grammarly® software while the control group did not receive any treatment. The treatment phase below describes the teaching/learning behaviors in each group.
Intervention Phase
In the experimental sub-groups at the two proficiency levels, the students were trained to make use of the Grammarly® software and keep records of their performances. They were also briefed about various writing strategies through pamphlets and lectures. To apply the treatment, the subjects in GCG were asked to develop 8 writing tasks all in 150-190 words during the treatment using the software. The tasks were given one by one as assignments, each task at the beginning of a week, and the due date of submission was by the end of the same week. After their tasks were completed, they uploaded their writings on the webpage (https://www.grammarly.com/grammar-check), a free version of Grammarly®, to receive instant feedback through this software. The finished products were collected and analyzed every week by the teacher and the participants were also permitted to send their final drafts to the teacher through email or WhatsApp before the next session of the classroom.
In the control sub-groups, TCG, the participants were asked to develop 8 writing tasks all in 150-190 words in each week and received feedback from the teacher after the completion of the writing task. The participants were asked to write about the topics that were determined for them by the researcher in each week. The participants were asked to hand in their writings by the end of the same week. The teacher corrected the learners’ papers and provided them with the appropriate feedback required and then sent the scanned papers to them via email or WhatsApp application before the next classroom session. This process continued until the eighth session.
Post-treatment Phase
After the eighth writing session which was the closing step for the treatment, the participants were asked to take part in the posttest of writing to see how well they had improved their writing skills after they had experienced the treatment. This way the participants’ writing skills in the four areas were evaluated by a posttest in the same way as the pretest. In addition, the learners in the treatment sub-groups at the lower and upper proficiency levels received the writing attitude questionnaire as the posttest. Likewise, the same learners were interviewed in terms of their views concerning the method through which they had been trained during the writing course.
Data Analysis
In the current study descriptive statistics and KR-21 reliability of Oxford Quick Placement Test were taken into consideration. Likewise, Pearson correlations were run to probe the inter-rater reliability indices of the two raters who rated the EFL learners on pretests and posttests of sub-skills of writing; i.e. task response, cohesion and coherence, grammatical range and accuracy, and lexical resources. In the next step, Cronbach’s alpha reliability indices were estimated for the writing attitude questionnaire. Moreover, two sets of two-way analysis of covariance (Two-way ANCOVA) and a multivariate analysis of covariance (MANCOVA) were used to probe the first three research questions. The last research question of the study required both quantitative and qualitative data analyses. Accordingly, frequencies, percentages and standardized residuals; attitude towards Grammarly ® by proficiency levels were calculated and then, a thematic analysis was conducted to analyze the interview data.
RESULTS
Exploring Research Question One and Two
The first two research questions targeted the effect of immediate corrective feedback provided by Grammarly® software on lower and upper intermediate EFL learners’ writing ability. A Two-way ANCOVA was run to compare the treatment and control lower and upper intermediate EFL learners’ mean scores on the posttest of writing after controlling for the effect of pretest. The results of the Two-way ANCOVA were followed by simple effect analyses in order to compare treatment and control groups at two proficiency levels.
Table 3 displays the experimental and control groups’ means on posttest of writing after controlling for the effect of pretest. The results indicated that the experimental group (M = 74.82, SE = .40) had a higher mean than the control group (M = 51.84, SE = .40) on posttest of writing after controlling for the effect of pretest.
Table 3 Descriptive Statistics; Posttest of Writing by Groups with Pretest | ||||
Groups | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | |||
Experimental | 74.828a | .400 | 74.032 | 75.624 |
Control | 51.847a | .400 | 51.051 | 52.643 |
a. Covariates appearing in the model are evaluated at the following values: Pre-Writing = 50.83. | ||||
Table 4 displays the lower and upper intermediate groups’ means on posttest of writing after controlling for the effect of pretest. The results indicated that the upper intermediate group (M = 68.94, SE = .40) had a higher mean than the lower intermediate group (M = 57.73, SE = .40) on posttest of writing after controlling for the effect of pretest.
Table 4 Descriptive Statistics; Posttest of Writing by Proficiency Levels with Pretest | ||||
Proficiency Levels | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | |||
Upper Intermediate | 68.940a | .400 | 68.143 | 69.738 |
Lower intermediate | 57.735a | .400 | 56.937 | 58.532 |
a. Covariates appearing in the model are evaluated at the following values: Pre-Writing = 50.83. | ||||
The results displayed in Table 5; i.e. (F (1, 75) = 389.21, p < .05, partial η2 = .838 indicated a large effect size) indicated that the difference between the upper and lower intermediate groups’ means on posttest of writing (Table 5) after controlling for the effect of pretest was statistically significant. Based on the results, it can be concluded that the upper intermediate groups significantly outperformed the lower intermediate groups on posttest of writing after controlling for the effect of pretest.
Table 5 Tests of Between-Subjects Effects; Posttest of Writing by Groups by Proficiency Levels with Pretest | ||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared |
Pretest | 3206.665 | 1 | 3206.665 | 503.259 | .000 | .870 |
Group | 10498.005 | 1 | 10498.005 | 1647.574 | .000 | .956 |
Proficiency | 2479.966 | 1 | 2479.966 | 389.210 | .000 | .838 |
Group * Proficiency | 6.957 | 1 | 6.957 | 1.092 | .299 | .014 |
Error | 477.885 | 75 | 6.372 |
|
|
|
Total | 339403.000 | 80 |
|
|
|
|
Table 6 displays the descriptive statistics for the interaction between groups and proficiency levels on posttest of writing after controlling for the effect of pretest. The results of Two-way ANCOVA (F (1, 75) = 1.09, p > .05, partial η2 = .014 representing a weak effect size) showed that there was not any significant interaction between types of treatments and proficiency levels on posttest of writing after controlling for the effect of pretest. As displayed in Table 6, the experimental upper and lower intermediate groups had higher means than the control upper and lower intermediate groups.
Table 6 Descriptive Statistics; Posttest of Writing by Groups by Proficiency Levels with Pretest | |||||
Groups | Proficiency Levels | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | ||||
Experimental | Upper Intermediate | 80.133a | .565 | 79.008 | 81.258 |
Lower intermediate | 69.523a | .565 | 68.397 | 70.649 | |
Control | Upper Intermediate | 57.747a | .566 | 56.619 | 58.875 |
Lower intermediate | 45.947a | .572 | 44.808 | 47.086 | |
a. Covariates appearing in the model are evaluated at the following values: Pre-Writing = 50.83. | |||||
Based on the above results, it can be concluded that;
A: The experimental lower intermediate group (M = 69.52) significantly outperformed the control lower intermediate group (M = 45.94) on the writing posttest after controlling for the effect of pretest (MD1 = 23.57, p < .05).
B: The experimental upper intermediate group (M = 80.13) significantly outperformed the control upper intermediate group (M = 57.74) on posttest of writing after controlling for the effect of pretest (MD = 22.38, p < .05).
Exploring Research Question Three
A multivariate analysis of covariance (MANCOVA) was run to compare the experimental and control lower and upper intermediate groups’ means on posttests of writing skills after controlling for the effect of pretests (See Table 7). The results indicated that the experimental upper and lower intermediate groups had higher means than the control upper and lower intermediate groups on all four writing skills.
Table 7 Descriptive Statistics; Posttests of Writing Skills by Groups by Proficiency Levels with Pretests | ||||||
Dependent Variable | Group | Proficiency | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | |||||
Post-Task Response | Experimental | Upper Intermediate | 20.804a | .271 | 20.265 | 21.344 |
Lower intermediate | 17.556a | .260 | 17.037 | 18.074 | ||
Control | Upper Intermediate | 14.736a | .258 | 14.222 | 15.250 | |
Lower intermediate | 12.604a | .298 | 12.010 | 13.197 | ||
Post-Cohesion Coherence | Experimental | Upper Intermediate | 21.303a | .167 | 20.970 | 21.636 |
Lower intermediate | 19.067a | .161 | 18.747 | 19.387 | ||
Control | Upper Intermediate | 16.828a | .159 | 16.511 | 17.145 | |
Lower intermediate | 12.852a | .184 | 12.485 | 13.218 | ||
Post-Grammatical Range Accuracy | Experimental | Upper Intermediate | 20.241a | .146 | 19.950 | 20.532 |
Lower intermediate | 17.466a | .140 | 17.187 | 17.745 | ||
Control | Upper Intermediate | 14.774a | .139 | 14.497 | 15.051 | |
Lower intermediate | 10.769a | .160 | 10.449 | 11.089 | ||
Post-Lexical Resources | Experimental | Upper Intermediate | 18.302a | .124 | 18.055 | 18.548 |
Lower intermediate | 15.203a | .119 | 14.966 | 15.440 | ||
Control | Upper Intermediate | 12.275a | .118 | 12.040 | 12.510 | |
Lower intermediate | 8.570a | .136 | 8.299 | 8.841 | ||
| ||||||
Based on the MANCOVA results, presented in Table 8, it can be concluded that a) there were significant differences between the experimental and control groups’ means on posttests of writing skills after controlling for the effect of pretests (F (4, 69) = 1092.36, p < .05, pη2 = .984 representing a large effect size), b) there were significant differences between the upper intermediate and lower intermediate groups’ means on posttests of writing skills after controlling for the effect of pretests (F (4, 69) = 277.41, p < .05, pη2 = .941 representing a large effect size), and c) there were significant interactions between the types of treatments and proficiency levels on posttests of writing skills after controlling for the effect of pretests (F (4, 69) = 22.11, p < .05, pη2 = .562 representing a large effect size).
Table 8
Multivariate Tests; Posttests of Writing Skills by Groups by Proficiency Levels with Pretests
|
|
|
|
|
|
|
|
|
|
Effect | Value | F | Hypothesis df | Error df | Sig. | Partial Eta Squared |
Intercept |
|
| Wilks' Lambda | .029 | 580.015 | 4 | 69 | .000 | .971 |
PreTaskRe |
|
| Wilks' Lambda | .488 | 18.105 | 4 | 69 | .000 | .512 |
PreCohCoh |
|
| Wilks' Lambda | .766 | 5.265 | 4 | 69 | .001 | .234 |
PreGraAcc |
|
| Wilks' Lambda | .433 | 22.593 | 4 | 69 | .000 | .567 |
PreLexical |
|
| Wilks' Lambda | .856 | 2.909 | 4 | 69 | .028 | .144 |
Group |
|
| Wilks' Lambda | .016 | 1092.363 | 4 | 69 | .000 | .984 |
Proficiency |
|
| Wilks' Lambda | .059 | 277.410 | 4 | 69 | .000 | .941 |
Group * Proficiency Wilks' Lambda .438 22.114 4 69 .000 .562
| ||||||||
a. Covariates appearing in the model are evaluated at the following values: Pre-Task Response = 12.69, Pre-Cohesion Coherence = 13.51, Pre-Grammatical Range Accuracy = 13.39, Pre-Lexical Resources = 11.24.
Table 9 compares the lower and upper intermediate groups’ means on posttest of writing skills including task response, grammatical accuracy, lexical resource, and cohesion and coherence after controlling for the effect of pretests at individual levels of proficiency.
| ||||||||
Table 9 Pairwise Comparisons | ||||||||
Dependent Variable | Proficiency | (I) Group | (J) Group | Mean Difference (I-J) | Std. Error | Sig. | 95% Confidence Interval for Difference | |
Lower Bound | Upper Bound | |||||||
Post-Task Response | Upper Intermediate | Experimental | Control | 6.068* | .356 | .000 | 5.359 | 6.777 |
Lower intermediate | Experimental | Control | 4.952* | .399 | .000 | 4.156 | 5.748 | |
Post-Cohesion Coherence | Upper Intermediate | Experimental | Control | 4.475* | .220 | .000 | 4.037 | 4.913 |
Lower intermediate | Experimental | Control | 6.215* | .247 | .000 | 5.724 | 6.707 | |
Post-Grammatical Range Accuracy | Upper Intermediate | Experimental | Control | 5.467* | .192 | .000 | 5.084 | 5.849 |
Lower intermediate | Experimental | Control | 6.697* | .215 | .000 | 6.268 | 7.126 | |
Post-Lexical Resources | Upper Intermediate | Experimental | Control | 6.027* | .163 | .000 | 5.703 | 6.351 |
Lower intermediate | Experimental | Control | 6.633* | .182 | .000 | 6.269 | 6.997 | |
*. The mean difference is significant at the .05 level. | ||||||||
The findings displayed in Tables 7 through 9 indicated that;
A. The lower intermediate experimental group significantly outperformed the lower intermediate control group on posttest of four writing skills namely, task response, cohesion and coherence, grammatical range and accuracy and lexical resources.
B. The upper intermediate experimental group significantly outperformed the upper intermediate control group on posttest of four writing skills namely, task response, cohesion and coherence, grammatical range and accuracy and lexical resources.
Exploring Research Question Four
To answer this question both the results of the attitude questionnaire and interviews with learners in the experimental groups before and after the treatment were taken into consideration.
Writing Attitude Questionnaire
Table 10 displays the experimental group’s attitude towards the Grammarly® software prior to and after the utilizing the program. Based on the results displayed in Table 10, it can be concluded that the experimental groups after receiving Grammarly® showed more agreement with this program. The percentages of “agree” and “strongly agree” on the pretest were 10.2 and 1.3 percent respectively, while they increased to 18.5 and 10.7 on the posttest. The standardized results for these options were lower than -1.96, while they were higher than 1.96 on the posttest indicating that the percentages of agreement were significantly beyond what was expected.
Table 10 Frequencies, Percentages and Standardized Residuals; Attitude towards Grammarly® | ||||||||
| Choices | Total | ||||||
Strongly Disagree | Disagree | No Idea | Agree | Strongly Agree | ||||
| Pretest | Count | 207 | 260 | 199 | 77 | 10 | 753 |
% | 27.5% | 34.5% | 26.4% | 10.2% | 1.3% | 100.0% | ||
Std. Residual | 3.4 | 2.2 | -.7 | -3.0 | -5.3 |
| ||
Posttest | Count | 122 | 199 | 222 | 142 | 82 | 767 | |
% | 15.9% | 25.9% | 28.9% | 18.5% | 10.7% | 100.0% | ||
Std. Residual | -3.4 | -2.1 | .7 | 3.0 | 5.2 |
| ||
Total | Count | 329 | 459 | 421 | 219 | 92 | 1520 | |
% | 21.6% | 30.2% | 27.7% | 14.4% | 6.1% | 100.0% | ||
Table 11 displays the experimental lower and upper intermediate groups’ attitude towards the Grammarly® software after utilizing the program. Based on the results displayed in Table 11, it can be concluded that the lower and upper intermediate groups had almost the same attitude towards the Grammarly®. None of the standardized residuals were higher than +/- 1.96. That is to say; there were not any significant differences between the two groups’ selection of the choices.
Table 11 Frequencies, Percentages and Standardized Residuals; Attitude towards Grammarly ® by Proficiency Levels | ||||||||
| Choices | Total | ||||||
Strongly Disagree | Disagree | No Idea | Agree | Strongly Agree | ||||
| Lower Intermediate | Count | 58 | 92 | 106 | 63 | 43 | 362 |
% | 16.0% | 25.4% | 29.3% | 17.4% | 11.9% | 100.0% | ||
Std. Residual | .1 | -.2 | .1 | -.5 | .7 |
| ||
Upper Intermediate | Count | 64 | 107 | 116 | 79 | 39 | 405 | |
% | 15.8% | 26.4% | 28.6% | 19.5% | 9.6% | 100.0% | ||
Std. Residual | -.1 | .2 | -.1 | .5 | -.7 |
| ||
Total | Count | 122 | 199 | 222 | 142 | 82 | 767 | |
% | 15.9% | 25.9% | 28.9% | 18.5% | 10.7% | 100.0% | ||
Interviews
To collect the qualitative data, the interview guide was used. Accordingly, 20 participants were interviewed in each experimental sub-group following the intervention process for their attitudes towards the method that the researcher used in their class.
The interviews were conducted mainly in English. However, in case the learner preferred to speak Farsi, it was welcomed. Open coding (general related views) and axial coding (specific issues) were utilized to analyze and categorize the data. The findings of the interviews with regard to the lower and upper intermediate level students’ posterior views to the intervention process they had taken were as follows.
Item One:
What’s your idea about the method that was used in the classroom during this term? (Using the Grammarly® software in the writing class).
Table 12 shows the results obtained from the preliminary analysis of each experimental sub-groups among 20 EFL learners that presented their ideas about the method they had experienced. As the table below illustrates, all of those who were interviewed (100%) in both subgroups reported that they had liked the warm atmosphere in the class. Considering motivation, the amount of motivation was 85 % for the lower Intermediate (LI) group while all the upper Intermediate (UI) group learners (100%) expressed the fact that they were motivated in the classroom. Similar feelings were reported by both groups in terms of students' being engaged in the classroom conversations and participating in the classroom activities. More to the point, students had to read a lot outside the classroom and practice grammar and vocabulary in order to play an important role in L2 writing. 100 % of the UI group had found the Grammarly® software highly interactive, while 70% of the LI group had mentioned this notion. Finally, 25% of the LI group had found the Grammarly ® software boring, while for the UI group, it was 10 %.
Table 12
Participants’ Viewpoints about Using Grammarly® Software in the L2 Writing Class
No. |
Open Codes |
Axial codes | Percentage | |||
UI | LI | UI | LI | |||
1 |
Feelings and Motivation | The class enjoyed a warmly atmosphere. | 20 | 20 | 100% | 100% |
2 | The amount of motivation was high, especially in the classroom. | 20 | 17 | 100% | 85% | |
3 | Students were involved in the classroom activities. | 20 | 20 | 100% | 100% | |
4 | Students had to read a lot and practice grammar and vocabulary outside the classroom to play a significant role in L2 writing. | 20 | 20 | 100% | 100% | |
5 | The Grammarly software was highly interactive | 20 | 14 | 100% | 70% | |
7 | The Grammarly software was boring. | 2 | 5 | 0.10% | 25% | |
Item Two:
How well the immediate feedback provided by the software could be useful for you?
The views presented by the participants in the two groups regarding their improvement in L2 writing through the feedback provided by Grammarly® are shown in Table 13. Based on the results categorized in Table 13, among the five components indicated by the students the UI group learners could gain a minor priority over their LI counterparts. Meanwhile, the differences are not notable. In other words, the students in both experimental subgroups improved their writing skills by the feedback provided by the software.
Table 13
Students’ Viewpoints about the Immediate Feedback Provided by the Software on L2 Writing
1.Feeling about writing in English | Very Confident
| Confident
| Not Confident
|
UI (70 %) | UI (30 %) | UI (0.00 %) | |
LI (60%) | LI (30%) | LI (10%) | |
2.The amount of effort one makes on writing assignments | Significant Effort
| Appropriate Effort
| Inadequate Effort
|
UI (80 %) | UI (10 %) | UI (10 %) | |
LI (70%) | LI (10%) | LI (20%) | |
3.Understanding the feedback on the assignment | Mostly Understand
| Somewhat Understand
| Inadequately Understand
|
UI (80 %) | UI (20 %) | UI (0.00 %) | |
LI (80%) | LI (20%) | LI (0.00%) | |
4. Understanding the Grammarly® software's comments on the assignment. | Mostly Understand
| Somewhat Understand
| Inadequately Understand
|
UI (90 %) | UI (10 %) | UI (0.00 %) | |
LI (80%) | LI (10%) | LI (10%) | |
5. Ability to correct mistakes using the feedback from the Grammarly® software | Yes | Maybe | No |
UI (80 %) | UI (20 %) | UI (0.00 %) | |
LI (70%) | LI (10%) | LI (20%) |
Item Three:
Which writing errors were more emphasized in the Grammarly® software based on your experience? (Grammar, lexicon, cohesion/coherence, task response).
Table 14 shows the views presented by the participants in the two groups regarding their ability to evaluate their writing skills and its components. As the table displays, the learners’ preferences of kinds of errors to be concentrated on are important. The results revealed that;
A. Both experimental subgroups tend to focus on grammatical errors as much as possible (100%)
B. In contrast to the LI (80%), UI group (100%) preferred to focus on lexical resource errors
C. The LI group preferred task response by (80%) more than the UI group (60%).
D. The UI group (80%) and LI group (80%) similarly focused on cohesion and coherence errors.
Table 14 Errors More Emphasized in the Grammarly® Software Based on Learners' Experience | ||||
| Methods | |||
UI | LI | |||
Errors to be concentrated on assessing L2 Writing in the process of self-assessment | Grammatical Accuracy | N | 20 | 20 |
% | 100% | 100% | ||
Lexical Resource | N | 20 | 16 | |
% | 100% | 80% | ||
Task Response | N | 12 | 16 | |
% | 60% | 80% | ||
Cohesion & Coherence | N | 16 | 16 | |
% | 80% | 80% | ||
|
|
|
|
|
Item Four:
Do you think that your interactions with the Grammarly® software was successful? In what respects?
Table 15 provides the students’ viewpoints of the two groups regarding their interactions with the Grammarly® software.
Table 15
Students’ Viewpoints about their Interactions with the Grammarly® Software
1.Taking the responsibility of learning | Much to Very Much
| Moderately
| Little to A little
|
UI (80 %) | UI (20 %) | UI (0.00 %) | |
LI (70%) | LI (10%) | LI (20%) | |
2.Feeling autonomous in learning and promotion of the power of learning | UI (80 %) | UI (10 %) | UI (10 %) |
LI (60%) | LI (30%) | LI (10%) | |
3. Accurate mastery of language forms | UI (80 %) | UI (10 %) | UI (10 %) |
LI (70%) | LI (20%) | LI (10%) | |
4. Application of learned material to new contexts | UI (80 %) | UI (10 %) | UI (10 %) |
LI (70%) | LI (20%) | LI (10%) | |
5. Understanding of language rules | UI (80 %) | UI (10 %) | UI (10 %) |
LI (80%) | LI (10%) | LI (10%) | |
6.Facilitating the learning process | UI (90 %) | UI (10 %) | UI (0.00 %) |
LI (70%) | LI (10%) | LI (20%) |
Concerning one of the most significant factors in learning L2 writing namely, taking the responsibility of learning, 80% of those who were interviewed commented that they were able to take the responsibility of their learning. Similarly, 70% of the LI learners indicated the same idea. Likewise, the majority of learners in the UI group learners (80%) and 60% of the learners in the LI group felt they could be autonomous and improve their language learning skills. As to the precise mastery of language forms, 80% of UI group learners and 70% of LI group felt that communications had helped them to increase their mastery on the language forms. Furthermore, 80% of UI group learners and 70% of LI group learners felt that their connection with the Grammarly® software had helped them to increase their ability to use the acquired knowledge in the new circumstances. Remarkably, only 10% of those who participated in this study in both group confirmed this idea. Concerning the understanding language rules, only a small number of participants (10%) in both group commented that Grammarly® helped them to grasp language rules well, while 80% of UI and LI who participated in this study confirmed this idea; moreover, only 10% of LI group learners indicated that using Grammarly® had not facilitated the process of language learning rules, while 90% of UI group learners and 70% of LI group learners who participated in this study confirmed this idea.
Item Five:
Would you like to say anything about the method that had been used in the semester just finished?
Table 16 below provides some important points mentioned by the students who were interviewed regarding the positive facts of the method that they experienced in the current study.
The students of the two groups who were interviewed mentioned some significant points concerning the positive points of the method they experienced in their instructional type in the current study. They are presented in Table 16 below.
Table 16 Students’ Views about the Method Just Received | |||||
No. | Views | Frequency (f) | Percentage | ||
1 |
| ENG. | LI | ENG. | LI |
2 | The software was very user friendly | 10 | 8 | 100% | 80% |
3 | Motivating students to go on | 9 | 7 | 90% | 70% |
4 | Using technology as a teaching aid | 8 | 7 | 80% | 70% |
5 | Emphasizing L2 writing | 10 | 8 | 100% | 80% |
6 | Making students work hard | 10 | 10 | 100% | 100% |
Most of UI learners mentioned Grammarly® compared to the traditional methods and writing classes that they had experienced provided a warm atmosphere among learners in class and they liked it. Also, the knowledge and data they received during this term was more than what they had experienced in the prior term. Moreover, as they commented their classroom learning was connected to their extracurricular pedagogical events and activities because they were highly motivated in their classroom. Nonetheless, because the assignments and activities throughout the term were beyond the ability of weak students in UI and LI groups, they indicated these activities were boring and they had not liked them.
DISCUSSION
With respect to the effect of computer-based feedback through Grammarly® on the Iranian learners’ L2 writing, it was revealed that the application of the software had a significant and positive effect on both upper and lower intermediate EFL learners' L2 writing. However, the upper intermediate EFL learners were significantly more benefited than the lower intermediate group from the immediate corrective feedback provided by Grammarly® software. Likewise, in terms of the writing skills development, the four main L2 writing skills had developed in both groups under the effect of using Grammarly®. Moreover, the findings of the research indicated that the teacher feedback had a positive effect on reducing language learners’ writing errors.
The current findings, based on the positive effect of technologically supported training of EFL learners on their L2 writing skills are in line with some of the previous studies such as Gulley's (2003) study on using Grammarly® software glosses on L2 text comprehension and vocabulary learning and Ruetten's (2003) study on developing composition skills. Both of these studies as early attempts made to employ Grammarly® in the EFL classroom indicated that the feedback presented through an intelligent and interactive system could both save time and encourage the learners to improve their L2 writing skills. Technology may have affected the writing skills negatively. These findings, however, are not consistent with Brockbank and McGill (1998) in which the learners were flustered when the amount of feedback provided by software was high. Undue feedback for summative tasks provided by Grammarly® may increase the number of errors. Indeed, some experts believed the amount of feedback provided by Grammarly®, especially for grammatical accuracy and lexical precision could be too much for students. This current issue influenced the students’ confidence when the learners felt that they made a lot of errors in a text (O’Neill & Russell, 2019). Today everyone has access to technology everywhere but this very ease of access to information by technology can decrease the level of critical thinking and increase the level of flaccidity among students. Moreover, using technology such as automated correction errors software causes the language learners to work individually without any guidance leading to misinterpretation and the lack of ability in critical thinking. Consequently, all these factors may have a negative impact on students’ ability to improve their writing skills.
The study findings also showed that teacher-correction had also a significant effect on improving the L2 writing of Iranian EFL learners. In line with findings of the current study in terms of the success of TCF in the improvement of L2 writing among EFL learners, Thornbury (1999) believed that Teacher Corrective Feedback (TCF), especially in the form of written feedback could be helpful for the learners to minimize their L2 weaknesses. Paulus (1999) also found that both peer and teacher feedback significantly affected the students’ writing accuracy. These findings, however, are against the findings of Amrhein and Nassaji (2010) in which they believed if an instructor modifies a repeated error, especially grammatical or lexical errors whenever it occurs, it decreases the level of self-correction in learners as they would not be held responsible for probing and revising their own errors and has a negative impact on language learners’ writing.
The current study findings, based on the positive effect of Grammarly® software on L2 writing of both upper intermediate and lower intermediate EFL learners are in line with the findings of some other studies such as Jean and Simard (2011) on grammar learning in English and French L2, and Sormunen's (2014) study on EFL grammar learning and teaching in Finland, all of which support the idea of immediate corrective feedback in increasing the grammatical accuracy of the L2 learners. These findings, however, are in contrast with O’Neill and Russell (2019) in which some problems such as inaccurate recommendations for users about summative tasks, a great deal of undue feedback provided by the software, misinterpretation of some grammatical errors, and misplacement lexical errors were generated by Grammarly® resulting in a negative impact on Academic Learning Advisors (ALAs) and the quality of the language learners’ writing.
In terms of learners' attitudes about their L2 writing skills development after utilizing the Grammarly® software, it was revealed that Grammarly® software had a positive impact on learners’ views about using this software. The current study findings based on EFL learners' attitudes about the immediate feedback provided by Grammarly® software on L2 writing are in line with O’Neill and Russell's (2019) study on university students’ perceptions of the automated feedback provided by Grammarly® in the Australasian context. The results of this study revealed that "students who received Grammarly® advice were significantly more likely to state that they had received a lot of useful feedback" (p. 51). Nonetheless, this study contradicts those of Hoang and Kunnan (2016). Regarding the grammatical accuracy of sentences, Grammarly® showed unsatisfactory functions in some studies (Hoang & Kunnan, 2016), especially when summative assessment tasks were considered (Perelman, 2017). The quality of automated tools function such as Grammarly® was lower than the function that was performed by human beings (Cheng, 2017; Dikli & Bleyle, 2014). Although the feedback provided by Grammarly® has a positive impact on students’ attitudes toward using automated software, some learners acknowledged Grammarly® problems that can influence their views toward using automated software. It is hard to escape the obvious conclusion that, both Grammarly® software CF and teacher CF are useful in helping EFL learners improve their writing skills. However, Grammarly® software showed to be more successful in the final analysis than the teacher corrective feedback in terms of helping the EFL learners to improve four writing skills namely, grammatical range and accuracy, lexical precision, task response, cohesion and coherence.
CONCLUSION
Based on the findings, it can be concluded that instruction through Grammarly® software has a positively meaningful impact on Iranian EFL learners’ L2 writing improvement. One of the contributing factors may have been the novelty of this method in the instructional environment for the EFL learners and the fact that the students need more time and opportunity to accommodate themselves to computer-based instructions. In addition, web-instruction as a means for ELT generally increases independent learning and would be helpful for autonomous learning of foreign language mostly for those with a high level of general language proficiency. Thus, it can be assumed that individuals with lower proficiency levels may require instructors’ face-to-face interaction and support more than the more proficient learners. In other words, learners may need a specific threshold level of language proficiency to make the most utilization of technology in terms of autonomously independent learning. Feedback on writing skills, whether it be from a teacher or a software, is extremely important in helping students to identify and correct any errors, inconsistencies, or misunderstandings in their writing. This can include feedback on grammar, spelling, punctuation, sentence structure, vocabulary, and more. By providing detailed and constructive feedback, teachers can help students to understand their strengths and areas for improvement, enabling them to make necessary adjustments to their writing and become more effective writers.
As traditional assessment (e.g. paper-and-pencil assessments) will be sooner or later replaced by computerized assessment methods, EFL learners should be familiarized through technology to cope with the new technologies in the area of ELT. Furthermore, for the implementation and development of effective pedagogy in technology programs, there is a dire need for both educators and learners to become active users in using technology and enhance their own language skills and strategies for choosing and managing technology materials. Automated feedback technologies are becoming increasingly popular and teachers should be invited to shift from the traditional methods to adopt new methods and strategies with the proper support and assistance especially in writing instruction.
It is evident that there is still much to be revealed about utilizing Grammarly® software in that the technology’s role in language teaching is solely a teaching assistance. What is of paramount importance is not the utilization of technology, but the quality of what is done with this medium of instruction. Thus, the utilization of Grammarly® software alone cannot be enough in the ELT classrooms. In fact, online or offline applications such as Grammarly® software, Microsoft word, and grammar checker software can be considered successful in helping learners improve their grammar knowledge and writing skills in English. It is also signified that learning via software and using facilities such as cellphones, computers, electronic notebooks would enhance human performance, which especially results in fewer significant errors in the medium-term and long-term periods (Lenkaitis, 2020). Such tools have also been made to assist individuals remember whatever they require to learn (Knag, 2021). More frequent retrieval of information or learned skills results in increasing the capacity of memory, too. Generally, the way the English learners can keep the skills and knowledge for the long term is a vital issue and can be achieved via computer applications and software.
According to Storch and Wigglesworth (2010), most practitioners, instructors, course designers, and others who participate in teaching have to cope with the challenge of forgetting skills in the medium or long-term. Teachers attempt to prepare appropriate pedagogical tools and materials to keep the students’ acquired knowledge and skills longer in their minds to reach a deeper cognitive processing. Such conclusions match previous studies suggesting computer applications and software as a writing-aided tool will develop the quality of language learners’ writing skill (Wang, Shang & Briody, 2013). Therefore, the feedback provided by teachers can be useful when the writing-aided software do not adequately help the learners to develop their writing skills. This situation leads to the workload of instructors becoming heavier in large classes. Considering that correcting student's work is a difficult and time-consuming process, a tool that automatically corrects the work, such as Grammarly® can help teachers correct minor mistakes. Grammarly® and other automated feedback tools, by taking on the role of an assistant, can free up time teachers for correcting sentences and allow more time for lesson planning, and it can also shorten the grading time for assignments as well. Given the above, language learners should view technology as an aid, enhancing their success in the process of language leaning.
The findings of the study offer pedagogical implications for English learning and teaching in general and use of Grammarly® on EFL learners' writing skills in particular. These outcomes may be used to redesign the writing activities at universities and other educational environments. The results can assist teachers, managers and material developers to design materials, syllabuses, textbooks and software in the domain of writing skills. More importantly, findings of the current research can encourage teachers to use technologies in their classes since, based on the learners’ views, new online applications such as Grammarly® assist language learners to develop their capabilities in writing.
References
Ahmadi, M. A. (2018). The use of technology in English language learning. International Journal of Research in English Education, 3(2), 115-125.
Alipour, F., Fathi, J., & Saeedian, A. (2019). Enhancing vocabulary learning and self-regulation via a mobile application: An investigation of the Memrise App. Journal of Modern Research in English Language Studies 5(1), 27-46.
Amrhein, H.R., & Nssaji, H. (2010). Written corrective feedback: What do students and teachers prefer and why? Canadian Journal of Applied Linguistics, 13(2), 95-127.
Belali, J., & Sadeghi, K. (2019). The role of corrective feedback timing in task engagement and oral performance. Applied Research on English Language, 9(2), 229-252.
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17(1). 34.
Brockbank, A., & McGill, I. (1998). Facilitating reflecting learning in higher education, Buckingham, England: Society for Research into Higher Education & Open University Press.
Chirimbu, S., & Tafazoli, D. (2013). Technology & media: Applications in language classrooms (TEFL, TESL & TESOL). Professional Communication and Translation Studies, 6(1-2), 187-194.
Cheng, G. (2017). The impact of online automated feedback on students' reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18–27.
Creswell, J. W., & Hirose, M. (2019). Mixed methods and survey research in family medicine and community health. Family Medicine and Community Health, 7(2), 1-6.
Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage publications.
Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17.
El Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2), 121-142.
Esfahani, F., Hashemifardnia, A., & Namaziandost, E. (2018). The effect of using WhatsApp on Iranian EFL learners’ vocabulary learning. Journal of Applied Linguistics and Language Research, 5(3), 256-267.
Ellis, R. (2009b). A typology of written corrective feedback. English Language Teaching, 66, 97-107. doi: 10.1093/elt/ccn023.
Ghahari, SH., Piruznejad, M. (2016). Recast and explicit feedback to young language learners: impacts on grammar uptake and willingness to communicate. Issues in Language Teaching (ILT),5(2), 187-209.
Ghafoori, B., Dastgoshadeh, A., Aminpanah, A., & Ziaei, S. (2016). The effect of CALL on Iranian EFL learners’ grammar of writing. International Journal of Language Learning and Applied Linguistics World, 12(3), 14-23.
Ghufron, M. A., & Rosyida, F. (2018). The role of Grammarly in assessing English as a Foreign Language (EFL) writing. Lingua Cultural, 12(4), 395-403.
Gulley, D. S. (2003). Grammarly® software glosses and their effect on L2 text comprehension and vocabulary learning. Language learning & Technology, 13(2), 48-67.
Hoang, G. T. L., & Kunnan, A. J. (2016). Automated essay evaluation for English language learners: A case study of MY Access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. Journal of Second Language Writing, 8, 1-10.
Hazarika, Z. (2017). Exploring the impact of technology in teaching English: TESOL In the context. European Journal of English Language and Literature Studies, 5(10), 19-28.
Jahangard, A., Rahimi, A., Norouzizadeh, M. (2020). Students’ attitudes towards computer-assisted language learning and its effect on their EFL writing. International Journal of Learning and Teaching. 12(3), 14-22.
Jean, G., & Simard, D. (2011) Grammar learning in English and French L2: Students’ and teachers’ beliefs and perceptions. Foreign Language Annals, 44(3), 465-492.
Kang, T. (2021). The effect of using technology in l2 listening: A meta-analysis of studies from 1980 to 2020. In V. Turel (Ed.), Design solutions for adaptive hypermedia listening software (pp. 1-30). IGI Global.
Kia Heirati, J., & Ahmadi Alashti, L. (2015). Attitudes toward using the Internet for language learning: A case of Iranian English teachers and learners. International Journal of Research Studies in Educational Technology, 4(1), 63-78.
Lenkaitis, C. A. (2020). Valuing technology in the L2 classroom: Student and teacher perceptions, preferences, and digital identity. In M. Kalantzis & B. Cope (Eds.), Handbook of research on digital learning (pp. 173-189). IGI Global.
Li, S. (2018). What is the ideal time to provide corrective feedback? replication of li, Zhu & Ellis (2016) and Arroyo & Vilmaz (2018). Language Teaching, 53(1), 1-13.
Marzban, A., Mojtahedzadeh, S.S. (2014). Collaborative negotiated feedback versus teacher-written feedback: impact on Iranian intermediate EFL learners‟ writing. Theory and Practice in Language Studies, 4(2), 293-302.
Merriam-Webster Dictionary (2016). Merriam-Webster, Inc.: New York, NY, USA.
Motallebzadeh, K., & Nematizadeh, S. (2011). Does gender play a role in the assessment of oral proficiency? English Language Teaching, 4(4), 165-172.
O’Neill, R., & Russell, A. M. T. (2019). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1), 42-56.
Parra G., L., & Calero S., X. (2019). Automated writing evaluation tools in the improvement of writing skills. International Journal of Instruction, 12(2), 209-226.
Patel, C. (2013). Use of multimedia technology in teaching and learning communication skill: An analysis. International Journal of Advancements in Research & Technology, 2(7), 116-123.
Paulus, M., T. (1999). The effect of peer and teacher feedback on student writing. Journal of Second Language Writing, 8(3), 256-289.
Perelman, L. (2017). Automated essay scoring and NAPLAN: A summary report. Retrieved from https://www.nswtf.org.au/files/automated_essay_scoring_and_naplan.pdf.
Rafanello, D. (2008). Writing well: It’s all about attitude. Professional Writing, 2(3), 58-60.
Ruetten, M. K. (2003). Developing composition skills: Rhetoric and grammar. Boston, Heinle.
Qassemzadeh, A., & Soleimani, H. (2016). The impact of feedback provision by Grammarly software and teachers on learning passive structures by Iranian EFL learners. Theory and Practice in Language Studies, 6(9), 1884-1894.
Sahragard, R., & Sadri, M. (2009). On the relationship between multiple intelligences, vocabulary learning knowledge, and vocabulary learning strategies among the Iranian EFL learners. The Iranian EFL Journal Quarterly, 3(82), p.18.
Schachter J, (1991). Corrective feedback in historical perspective. Interlanguage studies bulletin, 7(2), 89-102.
Schraudner, M. (2013). The online teacher’s assistant: Using automated correction programs to supplement learning and lesson planning. Taiwan: Asia University.
Sheen, Y. (2007). The effect of focused written corrective feedback and language aptitude on ESL learners’ acquisition of articles. TESOL Quarterly, 41, 255–283.
Shokrpour, N., Fallahzadeh, M.H. (2007). A Survey of the Students and Interns EFL Writing Problems in Shiraz University of Medical Sciences. The Asian EFL Journal, 9(1), 147-163.
Sormunen, M. (2014). Opinions about EFL grammar learning and teaching: A study of Finnish upper-secondary-school students. Master’s Thesis, University of Jyväskylä.
Thornbury, S. (1999). How to teach grammar. Harlow, Pearson Education.
Zhang, Z., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36(1), 90-102.
Biodata
Mohsen Ashrafganjoe is a Ph.D. candidate at Maybod Islamic Azad University. He is an instructor at Chamran University of Kerman and has been teaching English since 2017. He has published one book as well as some papers in international journals. His areas of interest in research are teaching methodology and CALL.
Email: mohsenashrafganjoe@gmail.com
Mohammad Javad Rezai is an associate professor of Applied Linguistics at Yazd university. His research interests include SLA, Psycholinguistics and Language Assessment.
Email: mrezai@yazd.ac.ir
Seyedeh Elham Elhambakhsh is an assistant professor at Yazd University. She has published three books and several papers in national and international journals. Her main areas of research interest include EAP/ESP, critical discourse analysis, textual analysis, SFL, sociolinguistics, and SLA/FLA.
Email: e.elhambakhsh@gmail.com
