The comparative role of paper-based and computer-delivered IELTS in the cognitive and meta-cognitive strategies use of Iranian IELTS candidates in the academic module reading
محورهای موضوعی : Curriculum Design and Development
Nahid Majidi
1
(
English Department, Ahar Branch, Islamic Azad University, Ahar, Iran
)
Elham Kavandi
2
(
English Language Teaching Department, Farhangian University
)
Mohammadreza khodadoust
3
(
Assistant Professor, Department of English Language Teaching, Farhangian University
)
کلید واژه: Cognitive strategies, Computer-delivered IELTS, IELTS, Metacognitive strategies, Paper-based IELTS, Reading test-taking strategies,
چکیده مقاله :
The present study examined the comparability of Paper-Based (PB) and Computer-Delivered (CD) IELTS in the academic module reading section, focusing on Iranian IELTS candidates' cognitive and metacognitive strategies. The study intended to determine if the delivery mode had any impact on the use of these strategies. To this aim, 200 upper-intermediate learners were randomly selected and divided into two groups to participate in the study under the two aforementioned test conditions. They completed a test-taking strategy survey under both test conditions. Moreover, five participants from each test condition group took part in think-aloud protocols. The quantitative data were analyzed by means of Independent-sample t-test and Multivariate Analysis of Variance (MANOVA) to test the research hypotheses. Content analysis of the think-aloud protocols was also conducted to identify the strategies employed by the IELTS candidates in both PB and CD formats. The results revealed that the PB group had a significantly higher mean than the CD group on the cognitive and metacognitive processes used in academic IELTS. However, the think-aloud protocols indicated that, in many cases, these differences were minimal, with cognitive and metacognitive processes being similarly employed across both formats. Moreover, significant differences were observed between the PB and CD groups in their reading test-taking strategies. The implications of these findings for test preparation and design are discussed.
The present study examined the comparability of Paper-Based (PB) and Computer-Delivered (CD) IELTS in the academic module reading section, focusing on Iranian IELTS candidates' cognitive and metacognitive strategies. The study intended to determine if the delivery mode had any impact on the use of these strategies. To this aim, 200 upper-intermediate learners were randomly selected and divided into two groups to participate in the study under the two aforementioned test conditions. They completed a test-taking strategy survey under both test conditions. Moreover, five participants from each test condition group took part in think-aloud protocols. The quantitative data were analyzed by means of Independent-sample t-test and Multivariate Analysis of Variance (MANOVA) to test the research hypotheses. Content analysis of the think-aloud protocols was also conducted to identify the strategies employed by the IELTS candidates in both PB and CD formats. The results revealed that the PB group had a significantly higher mean than the CD group on the cognitive and metacognitive processes used in academic IELTS. However, the think-aloud protocols indicated that, in many cases, these differences were minimal, with cognitive and metacognitive processes being similarly employed across both formats. Moreover, significant differences were observed between the PB and CD groups in their reading test-taking strategies. The implications of these findings for test preparation and design are discussed.
Ackerman, R., & Lauterman, T. (2012). Taking reading comprehension exams on screen or on paper? A metacognitive analysis of learning texts under time pressure. Computers in Human Behavior, 28, 1816–1828. https://doi.org/10.1016/j. chb.2012.04.023.
Assiri, M. S. (2011). Test-taking strategy use on the reading section of the TOEFL iBT: A study of Arab ESL learners (doctoral dissertation), Oklahoma State University.
Bachman, L. F. (2005). Statistical analysis for language assessment (2nd ed.). Cambridge University Press.
Baker, L., & Beall, L. C. (2014). Metacognitive processes and reading comprehension. In S. E. Israel (Ed.), Handbook of research on reading comprehension (pp. 397-412). Routledge.
Baron, N. S. (2015). Words onscreen: The fate of reading in a digital world. Oxford University Press.
Berthold, M. (2011). Reliability of quick placement tests: How much faith can we place on quick paper or internet based placement tests. Australian Journal of Teacher Education, 35(6), 1-23.
Bicak, B. (2013). Scale for test preparation and test taking strategies. Educational Sciences: Theory and Practice, 13(1), 279-289.
Birch, B. (2002). English L2 reading: Getting to the bottom. London: Erlbaum.
Boo, J. (1997) Computerized versus paper-and-pencil assessment of educational development: Score comparability and examinee preferences (doctoral dissertation), University of Iowa.
Boulware-Gooden, R., Carreker, S., Thornhill, A., & Joshi, R. M. (2007). Instruction of meta-cognitive strategies enhances reading comprehension and vocabulary achievement of third-grade students. The Reading Teacher, 61(1), 70-77.
Brevik, L. M. (2019). Gamers, surfers, social media users: Unpacking the role of interest in English. Journal of Computer Assisted Learning, 35(5), 595-606.
Brown, K. G., Charlier, S. D., & Pierotti, A. (2012). E-learning at work: Contributions of past research and suggestions for the future. International Review of Industrial and Organizational Psychology, 27, 89-114.
Cartwright, K. B., Lee, S. A., Taboada Barber, A., DeWyngaert, L. U., Lane, A. B., & Singleton, T. (2020). Contributions of executive function and cognitive intrinsic motivation to university students' reading comprehension. Reading Research Quarterly, 55(3), 345-369.
Chan, S., Bax, S., & Weir, C. (2018). Researching the comparability of paper-based and computer-based delivery in a high-stakes writing test. Assessing Writing, 36, 32-48.
Chick, N. (2013). Metacognition. Vanderbilt university center for teaching. Retrieved 31/8/2021 from https://cft.vanderbilt.edu/guides-sub-pages/metacognition/.
Cho, S., Xu, Y., & Rhodes, J. A. (2010). Examining English language learners' motivation of, and engagement in, reading: A qualitative study. Reading, 10(2), 205-221.
Clinton, V. (2019). Reading from paper compared to screens: A systematic review and meta-analysis. Journal of Research in Reading, 42, 288–325. https://doi.org/10.1111/ 1467-9817.12269.
Conrad, R. M., & Donaldson, J. A. (2012). Continuing to engage the online learner: More activities and resources for creative instruction (Vol. 35). John Wiley & Sons.
Daguay-James, H., & Bulusan, F. (2020). Metacognitive strategies on reading English texts of ESL freshmen: A sequential explanatory mixed design. TESOL International Journal, 15(1), 20-30.
Delgado, P., & Salmerón, L. (2021). The inattentive on-screen reading: Reading medium affects attention and reading comprehension under time pressure. Learning and Instruction, 71, 101396.
Doughty, C. (2001).Cognitive underpinnings of focus on form. Cambridge University Press.
Du, W., & Ma, X. (2021). Probing what's behind the test score: Application of multi-CDM to diagnose EFL learners' reading performance. Reading and Writing, 34(6), 1441-1466.
Elekaei, A., Tabrizi, H. H., & Chalak, A. (2020). A study into the impact of the choice of cognitive and meta-cognitive strategies and podcasts on vocabulary gain and retention levels in the telegram-based e-learning context. Teaching English with Technology, 20(2), 98-117.
Fotovatian, S., & Shokrpour, N. (2014). Comparison of the efficiency of reading comprehension strategies on Iranian university students' comprehension. Journal of College Reading and Learning, 37(2), 47-63.
Fulcher, G., & Davidson, F. (2007). Language testing and assessment. Routledge.
Ghafournia, N., & Afghari, A. (2013). The interaction between reading comprehension cognitive test-taking strategies, test performance, and cognitive language learning strategies. Procedia-Social and Behavioral Sciences, 70, 80-84.
Ghaith, G. M. (2020). Foreign language reading anxiety and metacognitive strategies in undergraduates' reading comprehension. Issues in Educational Research, 30(4), 1310-1328.
Gopal, R., & Singh, C. K. S. (2020). Arising reading patterns in understanding literary texts. Studies in English Language and Education, 7(2), 407-420.
Grabe, W., & Jiang, X. (2013). Assessing reading. In A. J. Kunnan (Ed.), The companion to language assessment, V.1, (pp.185-200). Wiley & Blackwell.
Hemmye, C. L. (2004). Meta cognitive and learning strategies used by adult novice web-based students (doctoral dissertation), University of West Florida.
Hidayati, A. N., Ramalia, T., & Abdullah, F. (2021). Leveraging Skype-based webinars as an English language learning platform. AL-ISHLAH: Jurnal Pendidikan, 13(1), 10-20.
Hopfenbeck, T. N. (2017). Coping with the conflicts and consequences of high-stake testing. Assessment in Education: Principles, Policy & Practice, 24(4), 471-473, DOI:10.1080/0969594X.2017.1383040.
Huang, L. S. (2013). Cognitive processes involved in performing the IELTS speaking test: Respondents' strategic behaviors in simulated testing and non-testing contexts. IELTS Research Reports Online Series, 1(2), 1-51.
Huddleston, A. P., & Rockwell, E. C. (2015). Assessment for the masses: A historical critique of high-stakes testing in reading. Texas Journal of Literacy Education, 3(1), 38-49.
Jung, J. (2017). Effects of task complexity, glossing and working memory on L2 reading and L2 learning. Unpublished doctoral dissertation, University College London.
Khansir A.A., & Gholami Dashti, J. (2014). The effect of question-generation strategy on Iranian EFL learners' reading comprehension development. English Language Teaching, 7(4), 38-45.
Lan, Y. J. (2020). Immersion into virtual reality for language learning. Psychology of Learning and Motivation, 72, 1-26.
Larsson, A. K., Gullberg, S., & Lundevall, J. (2019). Blended learning, Active Learning
Classroom and information literacy in higher military education. Teaching & Learning, 3(1), 25-36.
Lim, H. (2020). Exploring the validity evidence of a high-stake, second language reading test: an eye-tracking study. Language Testing in Asia, 10(1), 1-29.
Lin, L., Lam, J. W. L., & Tse, S. K. (2019). Test takers' strategy use and L2 Chinese reading test performance in mainland China: A structural equation approach. Studies in Educational Evaluation, 60, 189-198.
Ma, J. Y., Kerulis, A. M., Wang, Y., & Sachdev, A. R. (2020). Are workflow interruptions a hindrance stressor? The moderating effect of time-management skill. International Journal of Stress Management, 27(3), 252-268.
Magnusson, C. G., Roe, A., & Blikstad‐Balas, M. (2019). To what extent and how are reading comprehension strategies part of language arts instruction? A study of lower secondary classrooms. Reading Research Quarterly, 54(2), 187-212.
Maican, M. A., & Cocoradă, E. (2021). Online foreign language learning in higher education and
its correlates during the COVID-19 pandemic. Sustainability, 13(2), 781.
Motallebzadeh, K., &Nematizadeh, S. (2011). Does gender play a role in the assessment of oral proficiency?. English Language Teaching, 4(4), 165-172.
Motlagh, E. D. (2021). The countervailing effect of language proficiency and cultural adaptation on the frequency of cognitive-metacognitive strategies among EFL readers. International Journal of Current Science Research and Review, 4(1), 45-52.
Nergis, A. (2013). Exploring the factors that affect reading comprehension of EAP students. Journal of English for Academic Purposes, 12, 1-9.
Newman, F., Couturier, L., & Scurry, J. (2010). The future of higher education: Rhetoric, reality, and the risks of the market. John Wiley & Sons.
Nielsen, J., & Landauer, T. K. (1993, May). A mathematical model of the finding of usability problems. In P. Croton & D. Sullivan (Eds.), Proceedings of the INTERACT'93 and CHI'93 conference on human factors in computing systems (pp. 206-213). New York: Longman.
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4). 219-225.
Poushaneh, K., & Berenj Foroush Azar, G. (2020). The effect of mental imagery strategy instruction on reading comprehension of EFL students. Curriculum Research, 1(2), 25-33.
Rapp, A. A., Bachrach, D. G., & Rapp, T. L. (2013). The influence of time management skill on the curvilinear relationship between organizational citizenship behavior and task performance. Journal of Applied Psychology, 98(4), 668–677. https://doi.org/10.1037/a0031733.
Rauber, A.S., & Gil, G. (2004). Feedback to grammar mistakes in EFL classes: A case study. Brasileira de Lingüística Aplicada, 4(1), 277-298.
Richards, J. (2008), Second language teacher education today. RELC Journal, 39,158-165.
Rokhaniyah, H., & Putra, O. V. (2021). Developing web-based online test system to boost IELTS academic reading score. English Review: Journal of English Education, 9(2), 235-244.
Rosenshine, B. V. (2017). Skill hierarchies in reading comprehension. In R. J. Spiro, B. C. Bruce, &W.F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 535-554). New York: Routledge.
Salmeron, ´ L., & Delgado, P. (2019). Critical analysis of the effects of digital technologies on reading and learning. Culture and Education, 31, 465–480. https://doi.org/ 10.1080/11356405.2019.1630958.
Samiei, F., & Ebadi, S. (2021). Exploring EFL learners' inferential reading comprehension skills through a flipped classroom. Research and Practice in Technology Enhanced Learning, 16(1), 1-18.
Schooler, J. W. (2002). Re-representing consciousness: Dissociations between experience and meta-consciousness. Trends in Cognitive Sciences, 6, 339–344. https://doi.org/ 10.1016/S1364-6613(02)01949-6.
Sheorey, R., & Mokhtari, K. (2001). Differences in the metacognitive awareness of reading strategies among native and non-native readers. System, 29(4), 431-449.
Smallwood, J., & Andrews-Hanna, J. (2013). Not all minds that wander are lost: The importance of a balanced perspective on the mind-wandering state. Frontiers in Psychology, 4, 441-452.
Smallwood, J., & Schooler, J. W. (2006). The restless mind. Psychological Bulletin, 132, 946–958. https://doi.org/10.1037/0033-2909.132.6.946.
Tobia, V., & Bonifacci, P. (2020). Look back at text or rely on memory? Efficacy of reading comprehension strategies in good and poor oral comprehenders. Journal of Research in Reading, 43(4), 536-555.
Wahyono, E. (2019). Correlation between students' cognitive reading strategies and reading comprehension. Jurnal Studi Guru dan Pembelajaran, 2(3), 256-263.
Weir, C., Yan, J., O'Sullivan, B., & Bax, S. (2007). Does the computer make a difference?: The reaction of candidates to a computer-based versus a traditional hand-written form of the IELTS Writing component: effects and impact. International English Language Testing System (IELTS) Research Reports 2007, 7(1), 52-61.
Wistner, B., Sakai, H., & Abe, M. (2009). An analysis of the Oxford Placement Test and the Michigan English Placement Test as L2 proficiency tests. Bulletin of the Faculty of Letters, Hosei University, 58, 33-44.
Wolf, M. (2018). Reader, come home: The reading brain in a digital world. New York, NY: Harper.
Wu, A. D., Chen, M., & Stone, J. E. (2017). Investigating how test takers change their strategies to handle difficulty in taking a reading comprehension test: Implications for score validation. International Journal of Testing, 17(4), 1-23.
Zhang, Z., Yang, J., & Zhao, H. (2020). Retrospective reader for machine reading comprehension. National Key Research and Development Program of China, 12(2), 1-10.
Zulmaini, E. A. (2021). Teaching and learning process of test-taking strategies in answering reading comprehension section. ELT Forum: Journal of English Language Teaching, 10 (2), 113-124.
