(Dis)empowerment Role of Assessment in English Section of Iranian University Entrance Exam: EFL University Students Perspective in Focus

Subject Areas : Research in English Language Pedagogy

Mahbubeh Mahmudi-Nasrabadi ¹ ( English Department, Najafabad Branch, Islamic Azad University, Najafabad, Iran )
Omid Tabatabaei ² ( English Department, Najafabad Branch, Islamic Azad University, Najafabad, Iran )
Hadi Salehi ³ ( English Department, Najafabad Branch, Islamic Azad University, Najafabad, Iran )
Mahmud Mehrabi ⁴ ( English Department, Najafabad Branch, Islamic Azad University, Najafabad, Iran )

Received: 2020-10-22 Accepted : 2021-02-06 Published : 2021-01-01

Keywords: Messick’s Framework, validity, reliability, Assessment, University Entrance Exam,

Abstract :

The University Entrance Exam (UEE) in Iran, a high-stakes test in a multiple-choice form, has a significant effect on its stakeholders. This mixed-methods study investigates how freshman TEFL pupils experienced this nation-wide type of assessment and its (dis)empowerment role-taking advantage of Messick’s framework. One-hundred freshmen university TEFL students as participants of this study were conveniently and purposefully selected from Isfahan, Shiraz, and Tehran state universities. A validated questionnaire and group interviews were utilized to collect the data of this study. The data was analyzed and described through descriptive statistical procedures. Based on the results, Freshmen TEFL students considered the UEE mainly unreliable, while they held positive views towards its validity based on the modules of Messick’s framework. The overall analysis of the results pointed to the disempowerment role of the test. Furthermore, the obtained results underscored the necessity of more performance-based modes of assessment, such as using portfolios to be included in UEE. The results of the study bear useful implications for high-stake test constructors and policymakers.

References:

Ahmadi, G. (2004). Situation and role of educational improvement assessment in education and process-oriented learning. Paper presented at the first educational assessment symposium, Tehran: Ministry of Education.

Ahmadi, A., Darabi Bazvand, A., Sahragard, R., & Razmjoo, A. (2015). Investigating the Validity of PhD. Entrance Exam of ELT in Iran in Light of Argument-Based Validity and Theory of Action. Journal of Teaching Language Skills, 34(2), 1-37.

Aitken, N. (2012). Student voice in fair assessment practice. In C. F. Webber & J. L. Lupart (Eds.), Leading student assessment. Studies in educational leadership; ISSN 15 (pp. 175-200). Springer Netherlands.

Alderson, C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115-129.

Alibakhshi, G., & Ali, H. G. (2011). External validity of TOEFL section of Doctoral Entrance Examination in Iran: A mixed design study. Theory and Practice in Language Studies, 1(10), 1304-1310.

Barati, H., & Ahmadi, A. R. (2010). Gender-based DIF across the Subject Area: A Study of the Iranian National University Entrance Exam. The Journal of Teaching Language Skills (JTLS), 2(3), 1-22.

Bennett, R. E. (2010). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5-25.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Maidenhead UK: Open University Press.

Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud, & N. Falchikov (Eds.), Rethinking assessment in higher education: Learning for the longer term (pp. 14-25). Routledge.

Brown, J. D., & Hudson, T. (1996). The alternatives in language assessment. TESOL Quarterly,32, 653-675

Brown J. B. (1997) Textbook Evaluation Form. The Language Teacher,21 (10), 15-21.

Brown, R., Pressley, M., Van Meter, P. & Schuder, T. (1996). A quasi-experimental validation of transactional strategies instruction (with low-achieving second-grade readers, Journal of Educational Psychology, 88, 18-37.

Cassady, J. C. (2010). Test anxiety: Comtemporary theories and implications for learning. In J. C. Cassady (Ed.), Anxiety in schools: The causes, consequences, and solutions for academic anxieties, (pp. 5-26). Peter Lang.

Chapman, D. W., & Synder, C. W. (2000). Can high-stakes national testing improve instruction: reexamining conventional wisdom. International of Educational Development, 20, 427-474.

Cheng, L. (1997). How does washback influences teaching? Implications from Hong Kong. Language and Education, 11(1), 38-54. www.scholink.org/ojs/index.php/selt

Cheng, L. (2004). The washback effect of a public examination change on teachers’ perceptions toward their classroom teaching. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 146-170). Mahwah, Lawrence Erlbaum Associates.

Cheng, L. (2005). Changing language teaching through language testing: A washback study. Cambridge University Press.

Cheng, L., Y., Watanabe J., & Curtis, A. (2004). Washback in language testing: research contexts and methods. Mahwah, Lawrence Erlbaum.

Choi, I. (2008). The impact of EFL testing on EFL education in Korea. Language Testing, 25(1), 39-62.

Creswell, J.W. & Plano Clark, V.L. (2011) Designing and conducting mixed methods research. 2nd Edition, Sage Publications.

de Lange, J. (2007). Large-scale assessment and mathematics education. In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics (pp. 1111-1142). National Council of Teachers of Mathematics.

Derry, J. (2000). Iran. In D. Coulby, R. Cowen, & C. JONES (Eds). World yearbook of education 2000: Education in times of transition. Stylus Publications.

Erickson, G., & Åberg-Bengtsson, L. (2012). A collaborative approach to national test development. In D. Tsagari & I. Csépes (Eds.), Collaboration in
language testing and assessment (pp. 93–108). Peter Lang.

Farhadi, H. (1985). A study of English language test in national university entrance examination. Roshd Journal of Language Learning, 4, 10-15.

Farhady, H., Barati, H. (2009). Language assessment policy in Iran. Annual Review of Applied linguistics, 29, 131-141.

Freeman, D.J., Kuhs, T.M., Porter, A.C., Floden, R.E., Schmidt, W.H., & Schwille, J.R. (1983). Do textbooks and tests define a natural curriculum in elementary school mathematics. Elementary School Journal, 83(5), 501-513,

Fulcher, G., & Davidson, F. (2007). Language Testing and Assessment. Rotledge.

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference 11.0 update (4^th ed.). Boston: Allyn & Bacon

Ghorbani, M. R. (2008). Washback effect of the university entrance examination on Iranian pre-university English language teachers 'curriculum and instruction [Doctoral dissertation]. University of Putra Malaysia

Ghosgolk, A. (2005) A new look at educational improvement assessment and its role on educational reforms, Paper presented at first educational assessment symposium, Tehran: Ministry of Education.

Guion, R. M.(1976).Recruiting, selection, and job placement.In M. D.Dunnette (Ed.), Handbook of industrial and organizational psychology (pp.777-828). Rand McNally.

Gulliksen, H. (1950). Intrinsic validity. American Psychologist, 5,511-517.

Gumma Siddiek, A. (2010).Evaluation of the Sudan school certificate English examination. English Language Teaching Journal, 3(2), 37-47. http:// www.ccsenet.org/elt

Gustafsson, J. E., & Erickson, G. (2013). To trust or not to trust? Teacher marking versus external marking of national tests. Educational Assessment, Evaluation and Accountability, 25(1), 69-87.

Hasani, M. (2005) The handbook of implementing descriptive assessment for 1st, 2nd, and 3rd graders, Available at: http: //baher.blogfa.com/post-12.aspx.

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58(1), 47-77.

Hughes, A. (1989). Testing for Language Teachers. Cambridge University Press.

In'nami, Y., & Koizumi, R. (2011). Factor structure of the revised TOEIC test: A multiple-sample analysis. Language Testing, 29(1), 131-152

Johnson, R.B. & Onwuegbuzie, A.J. (2004) Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33, 14-26.
http://dx.doi.org/10.3102/0013189X033007014

Jokivuori, P., & Hietala, R. (2007). Määrällisiä tarinoita: Monimuuttujamenetelmien käyttö ja tulkinta. Porvoo: WSOY.

Kane, M. (1992). “An argument-based approach to validity.” Psychological Bulletin 112:527–535.

Khoii, R. (1998). A qualitative and quantitative evaluation of the English subtests of the entrance examinations of Universities using Rasch Model [doctoral dissertation]. Islamic Azad University: Science and Research Campus, Tehran.

Knekta, E. (2017). Are all pupils equally motivated to do their best on all tests? Differences in reported test-taking motivation within and between tests with different stakes. Scandinavian Journal of Educational Research, 61(1), 95–111.

Loumbourdi, L. (2014). The Power and Impact of Standardised Tests: Investigating the Washback of Language Exams in Greece. Peter Lang GmbH.

Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.

Madandar Arani, A. Farahbakhsh, S & Kakia, L. (2012) Primary schools in Japan and Iran: A comparative perspective on educational assessment system, In N. Popov, C. Wolhuter, B.Leutwyler, M. Mihova & J. Ogunleye (Eds). Comparative education, teacher training, education policy, social inclusion and history of education. Bureau for educational services.

Madaus, G. (1988). The influence of testing on the curriculum.” In J. Tanner (Ed.), The Influence of Testing on the Curriculum. University of Chicago Press.

Messick, S. (1989). “Validity.” In R. L. Linn (Ed.), Educational Measurement (3rd ed.), Phoenix, AZ: Oryx Press, pp. 13–103.

Metsämuuronen, J. (2009). Tutkimuksen tekemisen perusteet ihmistieteissä: Tutkijalaitos [The fundamentals of research in human sciences: The researcher
edition] (4th ed.). Helsinki: International Methelp.

Popham, W. J. (2001). Teaching to the test? Educational Leadership, 58(6), 16-20.

Sacks, P. (1999). Standardized minds: The high price of America's testing culture and what we can do change it. MA: Perseus Books.

Ravand, H., & Firoozi, T. (2016). Investigating Validity of UEE using the Rasch Model. International Journal of Language Testing, 6(1), 1- 23.

Saif, S. (2006). Aiming for positive washback: A case study of international teaching assistants. Language Testing, 23(1), 1-34.

Shih, C. (2009). How tests change teaching: A model for reference. English Teaching: Practice and Critique, 8 (2), 188-206

Shohamy, E. (2001). Democratic assessment as an alternative. Language Testing, 18(4), 373-391.

Shohamy, E. (2007). Tests as power tools: Looking back, looking forward. In J. Fox, M. Wesche, D. Bayliss, L. Cheng, C. Turner & C. Doe (Eds.), Language testing reconsidered (pp. 141-152). University of Ottawa Press

Spratt, M. (2005). Washback and the classroom: the implications for teaching and learning of studies of washback from exams. Language Teaching Research, 9, (1), 5–29.

Tsagari, D. (2011). Washback of a high-stakes English exam on teachers’ perceptions and practices. Selected Papers from the 19th ISTAL, 431- 445.

Virta, A. (2002). Arviointi oppimisen ja opetuksen punaisena lankana. In E. Lehtinen & T. Hiltunen (Eds.), Oppiminen ja opettajuus (pp. 63-86). Turun yliopisto. Kasvatustieteiden tiedekunnan julkaisuja B: 71.

Wall, D. (2000). The impact of high-stakes testing on teaching and learning. Can this be predicted or controlled? System, 28, 499-509.

Wall, D. (2005). The impact of high-stakes examinations on classroom teaching. Cambridge University Press.

Wolf, T.V., Rode, J.A., Sussman, J., Kellogg, W.A., 2006. Dispelling “design” as the black art of CHI. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '06. ACM, New York, NY, USA, pp. 521–530.

Xie, X. (2015). Methods of Test Validation. In E.Shohamy & N.Hornberger (Eds.). Language testing and assessment. Encyclopedia of language and education 7 (pp. 177-196). Springer Science + Business Media.

Yarmohammadi, L. (1986). A review of English language test of national university entrance examination year 1986. Shiraz University Journal of Social Sciences and Humanities, 2 (1), 80-88. In Persian.

Yildirim, O. (2010). Washback effects of a high-stakes university entrance exam: Effects of the English section of the university entrance exam on future English language teachers in Turkey. The Asian EFL Journal Quarterly, 12 (2), 92-116.

Zhang, G. (2014). Achievement gap in China. In J. V. Clark (Ed.), Closing the achievement gapfrom an international perpsective: Transforming STEM for effective education (pp. 217–228). Springer.