On the Consequential Validity of ESP Tests: A Qualitative Study in Iran
Subject Areas : آموزش زبان انگلیسی
غلامرضا کیانی
1
(Tarbiat Modares University)
گودرز علی بخشی
2
(Yasouj University)
رامین اکبری
3
(Tarbiat Modares University)
Keywords: validity, Language Teaching, Consequential Validity, ESP Tests,
Abstract :
Consequential validity, a component of construct validity as a unified concept introduced by Messick(1989), deals with the impacts of tests on teaching, learning, individual test takers, teachers, society, and educational system within a country. Although the impacts of language tests on teaching and learning have been somehow studied, the consequences of ESP tests on individual test takers and teachers have not been given appropriate attention. The present study aimed at exploring the possible consequences of ESP tests, as parts of master and doctoral entrance examinations to post graduate universities in Iran, on the stake holders and society. In doing so, a qualitative research method was applied. The participants were 31 master and doctoral students, and 5 ESP instructors. The data were analyzed in terms of the content. Four main themes with a variety of sub-themes emerged from the content analysis. The results indicated that ESP tests have negative psychological (anxiety, stress, disappointment, self-confidence, teaching efficacy, disappointment), social (deprivation from education, ethical issues, acceptance of nonqualified candidates), financial, and family consequences on the learners and teachers. The society is also influenced as the result of ESP tests scores.
Alderson, J. C. (2004). ‘Foreword’ in L. Cheng,Y. Watanabe, and A. Curtis (eds.). Washback in language testing: Research contexts and methods (pp ?). London: Lawrence Erlbaum.
Alderson, J.C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14 (2), 115–129
Alderson, J.C., & Hamp-Lyons, L.(1996). TOFEL preparation courses: a study of washback. Language Testing, 13(3), 280-97.
Andrews, S. (1994a). Washback or washout? The relationship between examination reform and curriculum innovation. In D. Nunan, R. Berry, & V. Berry, (Eds.) , Bringing about change in language education." Proceedings of the International Language in Education Conference 1994, (pp. 67-81). Hong Kong: University of Hong Kong.
.Andrews, S., & Fullilove, J. (1994). Assessing spoken English in public examination-why and how? In J. Boyle and P. Falvey (Eds.), English Language Testing in Hong Kong (pp. 57–86). Hong Kong: Chinese University.
Bachman, L. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press.
Bachman, L, & and Palmer, A. (1996). Language testing in practice. Oxford: Oxford University Press
Bailey, K. M. (1999). Washback in language testing. Educational testing service. Peicton: Newjersey
Chapman, D.W., & Snyder, C.W. (2000). Can high stakes national testing improves instruction: reexamining conventional wisdom. International Journal of Educational Development,20, 457–474.
Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong. Language and Education,11 (2), 38–54.
Cheng, L. (1998). Impact of a public English examination change on students’ perceptions and attitudes toward their English learning. Studies in Educational Evaluation 24 (3), 279–301.
Davies, A. (ed.) (1997). Special issue: ethics in language testing. Language Testing,14.
Foucault, M. (1979). Discipline and punish: The birth of the prison. [Tr. A. Sheridan.] New York: Vintage Books.
Hamp-Lyons, L. (1989). Language testing and ethics. Prospect, 5, 7-15.
Hamp-Lyons, L. (1997a). Ethics in language testing. In: Clapham, C. (Ed.), The Encyclopedia of Language and Education. Vol. 7. Language Testing and Assessment (pp. 323-333). Kluwer, Dordrecht.
Hamp-Lyons, L. (1997b).Washback, impact and validity: ethical concerns. Language Testing. 14 (3), 295-303.
Hamp-Lyons, L. (1999). Ethics, fairness(es), and developments in language testing. In: Elder, C., Brown, A., Grove, L., Hill, K., Lumley, T., McNamara, T. (Eds.), Experimenting with Uncertainty: Essays in Honor of Alan Davies; Studies in Language Testing 11 (pp. 212-217).UCLES/Cambridge University Press, Cambridge.
Hamp-Lyons, L. (2000). Social, professional and individual responsibility in language testing. System, 28, 579-591.
Herman, J.L., & Golan, S. (1993). The effects of standardized testing on teaching and schools. Educational Measurement: Issues and Practices, 12 (4), 20-25.
Hughes, A. (1988). Introducing a needs-based test of English language proficiency into an English-medium university in Turkey. In Hughes, A.(Ed.), Testing English for university study. ELT Documents 127 (pp.134-46). Oxford: Modern English Publications.
Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.
Kiani, R.G., Akbari, R., & Alibakhshi, G. ( In press). On the authenticity of ESAP test. Asian Journal of ESP.
Luxia, Q. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes test. Language Testing, , 22, 142-173
Madaus, G. (1990). Testing as social technology. Paper presented at the inaugural annual Boisi Lecture in Education and Public Policy. Boston College.
Marvdast, A. B. (2004). Qualitative research in sociology. London: SAGE publications.
McNamara, T. (1999). Validity in Language Testing: The Challenge of Sam Messick's Legacy. Messick Memorial Lecture, Language Testing Research Colloquium,
Messick, S. (1989). Validity. In: Linn, R.L. (Ed.), Educational Measurement (3rd Edition)(pp.13-103). American Council on Education/Macmillan, New York.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13-23.
---- 1996. Validity and washback in language testing. Language Testing, 13, 241-56.
Pearson, I. (1988). Tests as levers of change (or 'putting first things first'). In D. Chamberlain &
R. Baumgartner (Eds.), ESP in the classroom: Practice and evaluation (pp. 98-107). EL T Documents 128. London: Modem English Publications in association with the British Council.
Shohamy, E. (1993). The power of tests: the impact of language tests on teaching and learning. NFLC Occasional Papers.
Shohamy, E. (1997). Critical Language Testing and Beyond. Plenary address at the American Association of Applied Linguistics Conference.
Shohamy, E. (1998). Critical language testing and beyond. Studies in Educational Evaluation,24 (4), 331-345.
Shohamy, E. (2001). The Power of Tests. London: Longman/Pearson.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect overtime. Language Testing, 13(3), 298-317.
Spolsky, B. (1994). The examination-classroom backwash cycle: some historical cases. In Berry, R.(Ed.), Bringing about change in language education: proceedings of the International Language in Education Conference (pp. 55–66). Hong Kong: Department of Curriculum Studies.
Spolsky, B. (1997). The ethics of gate keeping tests: what have we learned in a hundred years? Language Testing,14 (3), 242-7.
Spolsky, B. (1981). Some ethical questions about language testing. In: Klein-Braley, C.,Stevenson, D. (Eds.), Practice and problems in language testing (pp.5-30). Verlag Peter D. Lang: Frankfurt.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage.
Taylor, L. (2005). Washback and impact. ELT Journal, 59(2).
Wall, D., (1997). Impact and washback in language testing. In: Clapham, C., Corson, D. (Eds.), Encyclopedia of Language and Education, Volume 7: Language Testing and Assessment (pp. 291–302). Kluwer, Dordrecht.
Wall, D., (2000). The impact of high-stakes testing on teaching and learning: can this be predicted or controlled? System, 28, 499–509.
Wall, D., & Alderson, J.C. (1993). Examining washback: the Sri Lankan impact study. Language Testing, 10, 41–69.
Watanabe, Y. (1997). Constructing a classroom observation scheme for the test impact research. Sophia Linguistica, 41, 297-313.
Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods. (pp.19-36). Mahwah, N.J.: Lawrence Erlbaum