The Effect of Paper versus Electronic Test on Iranian English Language Proficiency performance

Subject Areas : Educational management

Mojtaba Maghsoudi ¹ , Vahid Reza Mirzaeian ² , Maryam Roubdari ³

1 - Assistant Professor, English Department, Farhangian University, Tehran, Iran
2 - ELT Department, Faculty of Letters, Alzahra University
3 - Department of English Language, Islamic Azad University, Neka Branch, Iran

Received: 2020-02-17 Accepted : 2020-11-21 Published : 2020-12-21

Keywords: electronic test, English Language Proficiency, Paper test, Ph.D. student,

Abstract :

This research aimed at comparing the results gained in the two tests. The statistical population was 300 applicants taking part in MSRT. Only 75 had taken part in both tests and 60 of them agreed to share their test results with the researchers. The data was analyzed using descriptive statistics as well as inferential statistics. Results indicated a relatively high positive correlation between the total scores of both test (r = 0.71). Since no reports of test parts were available, it was impossible to compare the parts from the two exams. In addition, due to the omission of listening part in ETPNU test, this comparison was also not possible. As a result, the researchers decide to compare the listening section of MSRT with the ETPNU score (r = 0.38). Later, the other two sections of MSRT test were also compared with the ETPNU score (grammar, r = 0.33; reading comprehension, r = 0.53).

References:

[1]Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London, England: Continuum International Publishing. DOI: 10.1080/15434300701595637. [2]Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater v.2. Journal of Technology, Learning, and Assessment, 4(3), 1–31. https://ejournals.bc.edu/index.php/jtla/article/view/1650. [3]Barkaoui, K., Brooks, L., Swain, M., Lapkin, S. (2013). Test-takers’ strategic behaviors in independent and integrated speaking tasks. Applied Linguistics, 34, 304–324. https://doi.org/10.1093/applin/ams046. [4]Bates, D., Maechler, M., Bolker, B., Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01. [5]Burstein, J., & Chodorow, M. (2010). Progress and new directions in technology for auto-mated essay evaluation. In R. Kaplan (Ed.), The Oxford handbook of applied linguistics (2nd ed., pp. 529–38). Oxford, England: Oxford University Press. DOI: 10.1093/oxfordhb/9780195384253.013.0036. [6]Carr, N. T. (2006). Computer-based testing: Prospects for innovative assessment. In L. Ducate & N. Arnold (Eds.), Calling on CALL: From theory and research to new directions in foreign language teaching (CALICO monograph series, 5, pp. 289–312). San Marcos, TX: CALICO. DOI: 10.1111/j.1540-4781.2007.00639_23.x. [7]Chalhoub-Deville, M. (2010). Technology in standardized language assessments. In R. Kaplan (Ed.), The Oxford handbook of applied linguistics (2nd ed., pp. 511–26). Oxford, England: Oxford University Press. DOI: 10.1093/oxfordhb/9780195384253.013.0035. [8]Chapelle, C. A. (1998). Construct definition and validity inquiry in SLA research. In L. F. Bachman & A. D. Cohen (Eds.), Second language acquisition and language testing interfaces (pp. 32–70). Cambridge, England: Cambridge University Press. DOI: 10.1017/CBO9781139524711.004. [9]Chapelle, C. A. (2010). Technology in language testing[video]. Retrieved November 14, 2012 from http://languagetesting.info/video/main.html Chapelle, C. A., & Chung, Y.-R. (2010). The promise of NLP and speech processing technologies in language assessment. Language Testing, 27(3), 301–15. DOI: 10.1177/0265532210364405. [10]Chapelle, C. A., & Douglas, D. (2006). Assessing language through computer technology. Cambridge, England: Cambridge University Press. https://www.cambridge.org/us/cambridgeenglish/catalog/teacher-training-development-and-research/assessing-language-through-computer-technology. [11]Chapelle, C., Grabe, W., & Berns, M. (2000). Communicative language proficiency: Definition and implications for TOEFL 2000. TOEFL monograph series 10. Princeton, NJ: Educational Testing Service. https://www.ets.org/research/policy_research_reports/publications/report/1997/hzeu. [12]Coniam, D. (2006). Evaluating computer-based and paper-based versions of an English-language listening test. ReCALL, 18(2), 193–211. DOI: 10.1017/S0958344006000425. [13]Cotos, E. (2011). Potential of automated writing evaluation feedback. CALICO Journal, 28(2), 420–59. DOI: 10.11139/cj.28.2.420-459. [14]Crossley, S., Clevinger, A., Kim, Y. (2014). The role of lexical properties and cohesive devices in text integration and their effect on human ratings of speaking proficiency. Language Assessment Quarterly, 11, 250–270. https://doi.org/10.1080/15434303.2014.926905. [15]Cunnings, I. (2012). An overview of mixed-effects statistical models for second language researchers. Second Language Research, 28(3), 369–382. https://doi.org/10.1177/0267658312443651. [16]Dooey, P. (2008). Language testing and technology: Problems of transition to a new era. ReCALL, 20(1), 21–34 . DOI: 10.1017/S0958344008000311. [17]Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, England: Cambridge University Press. https://www.amazon.com/Assessing-Languages-Specific-Cambridge-Assessment/dp/0521585430. [18]Douglas, D. (2010). Understanding language testing. London, England: Hodder Education. https://www.amazon.com/Understanding-Language-Testing-Dan-Douglas/dp/0340983434 Douglas, D., & Hegelheimer, V. (2007). Assessing language using computer technology. Annual Review of Applied Linguistics, 27, 115–32. [20]  DOI: 10.1017/S0267190508070062. [19] Downey, R., Farhady, H., Present-Thomas, R., Suzuki, M., & Van Moere, A. (2008). Evaluation of the usefulness of the Versant for English test: A response. Language Assessment Quarterly, 5, 160–7. DOI: 10.1080/15434300801934744. [21] Fan, J., Jin, Y. (2019). Standards for language assessment: Demystifying university-level English placement testing in China. Asia Pacific Journal of Education. https://doi.org/10.1080/02188791.2019.1706445. [22]Freebody, P. (2013). Knowledge and school talk: Intellectual accommodations to literacy? Linguistics and Education, 24, 4–7. https://doi.org/10.1016/j.linged.2012.11.004. [23]Frost, K., Elder, C., Wigglesworth, G. (2012). Investigating the validity of an integrated listening-speaking task: A discourse-based analysis of test takers’ oral performances. Language Testing, 29(3), 345–370. https://doi.org/10.1177/0265532211424479. [24]Garrett, N. (1991). Technology in the service of language learning: Trends and issues. Modern Language Journal, 75, 74–101. DOI: 10.1111/j.1540-4781.1991.tb01085.x. [25]Henning, G., Hudson, T., & Turner, J. (1985). Item response theory and the assumption of unidimensionality for language tests. Language Testing, 2(2), 141–54. DOI: 10.1177/026553228500200203. [26]Jamieson, J. (2005). Trends in computer-based second language assessment. Annual Review of Applied Linguistics, 25, 228–42 www.cambridge.org/core/journals/annual-review-of-applied-linguistics/article/trends-in-computerbased-second-language-assessment/6C302FE25CCDB669A1984A6AE7B1D8E1. [27]Linck, J. A., Cunnings, I. (2015). The utility and application of mixed-effects models in second language research. Language Learning, 65(S1), 185–207. https://doi.org/10.1111/lang.12117. [28]Luoma, S. (2004). Assessing speaking. Cambridge language assessment series. Cambridge, England: Cambridge University Press. https://www.cambridge.org/us/cambridgeenglish/catalog/teacher-training-development-and-research/assessing-speaking/assessing-speaking-1. [29]Martin, J. R. (2013). Embedded Literacy: Knowledge as meaning. Linguistics and Education, 24, 23–37. [30]Maton, K. (2013). Making semantic waves: A key to cumulative knowledge-building. Linguistics and Education, 24, 8–22. https://doi.org/10.1016/j.linged.2012.11.005. [31] Maton, K. (2014). Knowledge and knowers. Towards a realist sociology of education. Oxon and New York: Routledge. Maton, K. (2016). Legitimation code theory: Building knowledge about knowledge-building. In Maton, K., Hood, S., Shay, S. (Eds.), Knowledge-building: Educational studies in Legitimation Code Theory (pp. 1–24). London: Routledge. [32]Matruglio, E., Maton, K., Martin, J. (2013). Time travel: The role of temporality in enabling semantic waves in secondary school teaching. Linguistics and Education, 24, 38–49. https://doi.org/10.1016/j.linged.2012.11.007. [33]Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. The Modern Language Journal, 93, 836–47 . DOI: 10.1111/j.1540-4781.2009.00976.x. [34]Parshall, C. G., Davey, T., & Pashley, P. J. (2000). Innovative item types for computerized testing. In W. J. Van der Linden & C. A. W. Glass (Eds.), Computerized adaptive testing: Theory and practice (pp. 129–48). Dordrecht, Netherlands: Kluwer. https://www.springer.com/gp/book/9780792364252. [35]Plakans, L. (2009a). Integrated assessment [video]. Retrieved: November 14, 2012 from http://languagetesting.info/video/main.html. [36]Plakans, L., Gebril, A. (2012). A close investigation of source use in integrated second language writing tasks. Assessing Writing, 17, 18–34. https://doi.org/10.1016/j.asw.2011.09.002. [37]Roever, C. (2006). Validation of a Web-based test of ESL pragmalinguistics. Language Testing, 23(2), 229–56. DOI: 10.1191%2F0265532206lt329oa. [38]Sawaki, Y. (2001). Comparability of conventional and computerized tests of reading in a second language. Language Learning & Technology, 5(2), 38–59. http://dx.DOI: 10125/25127. [39]Sawaki, Y., Stricker, L. J., & Oranje, A. H. (2009). Factor structure of the TOEFL Internet-based test. Language Testing, 26(1), 5–30. DOI: 10.1177%2F0265532208097335. [40]Weigle, S. C., Parker, K. (2012). Source text borrowing in an integrated reading/writing assessment. Journal of Second Language Writing, 21, 118–133. https://doi.org/10.1016/j.jslw.2012.03.004. [41]Winke, P. (2006). Online assessment of foreign language proficiency: Meeting development, design, and delivery challenges. In S. Howell & M. Hricko (Eds.), Online assessment and measurement: Case studies from teacher education, K-12 and corporate (pp. 82–97). London, England: Information Science Publishing. DOI: 10.4018/978-1-59904-935-9.ch180