Evaluating the validity of socially-situated assessment: Group dynamic assessment of intermediate EFL listening comprehension
محورهای موضوعی : Teaching
1 - English Department, Najafabad Branch, Islamic Azad University, Najafabad, Iran
کلید واژه: Group dynamic assessment, Micro-validity, Macro-validity, Learner L2 development, validity arguments, and Mediation,
چکیده مقاله :
This study evaluated the validity of group dynamic assessment (G-DA), grounded in Vygotsky’s (1987) Sociocultural Theory, implemented in a class of intermediate learners to assess and promote L2 listening comprehension. To navigate the dual goals of assessment and instruction, flexible mediation attuned to the zone of proximal development of the learners was provided within the G-DA interactions. This led to the detection of nine mediational strategies. The validity of these G-DA interactions was explored by extending Poehner’s (2011) validation model to classroom setting. Poehner’s (2011) model includes two interrelated foci for DA validation: micro and macro-validity. Following Kane's (2021) argument-based approach to validation, evidence-based arguments were developed to explore the appropriateness of each mediational strategy given to the learners (micro-validity) as well as the success of that mediational strategy and the entire G-DA procedure in promoting learners’ L2 listening comprehension (macro-validity). Class transcripts were analyzed to gain evidence for the micro- and macro-validity of the G-DA interactions. The findings supported the usefulness of Poehner’s validation model in developing validity arguments to determine the appropriateness of the interpretations made about learners’ abilities and the effects of the G-DA procedure on their development. Moreover, the study concluded that the analysis of learners’ independent performance needed to be added to Pohener’s macro-validation model so that it becomes applicable to G-DA
This study evaluated the validity of group dynamic assessment (G-DA), grounded in Vygotsky’s (1987) Sociocultural Theory, implemented in a class of intermediate learners to assess and promote L2 listening comprehension. To navigate the dual goals of assessment and instruction, flexible mediation attuned to the zone of proximal development of the learners was provided within the G-DA interactions. This led to the detection of nine mediational strategies. The validity of these G-DA interactions was explored by extending Poehner’s (2011) validation model to classroom setting. Poehner’s (2011) model includes two interrelated foci for DA validation: micro and macro-validity. Following Kane's (2021) argument-based approach to validation, evidence-based arguments were developed to explore the appropriateness of each mediational strategy given to the learners (micro-validity) as well as the success of that mediational strategy and the entire G-DA procedure in promoting learners’ L2 listening comprehension (macro-validity). Class transcripts were analyzed to gain evidence for the micro- and macro-validity of the G-DA interactions. The findings supported the usefulness of Poehner’s validation model in developing validity arguments to determine the appropriateness of the interpretations made about learners’ abilities and the effects of the G-DA procedure on their development. Moreover, the study concluded that the analysis of learners’ independent performance needed to be added to Pohener’s macro-validation model so that it becomes applicable to G-DA
Ableeva, R. (2010). Dynamic Assessment of listening comprehension in second language learning. Unpublished doctoral dissertation, The Pennsylvania State University, University Park, PA.
Aljaafreh, A., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal, 78(4), 465-483.
Bachman, L. F. (2000). Learner-directed assessment in ESL. In G. Ekbatani & H. Pierson (Eds.), Learner-directed assessment in ESL (pp. ix-xii). New Jersey: Lawrence Erlbaum Associates, Inc.
Bachman, L. F. (2005). Building and supporting a case for test use.Language Assessment Quarterly, 2(1), 1–34
Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.
Baird, J., Andrich, D., Hopfenbeck, T., & Stobart, C. (2017). Assessment and learning: Fields apart? Assessment in Education: Principles, Policy & Practice, 24, 317–350.
Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5–12.
Buck, G., (2001). Assessing listening. Cambridge: Cambridge University Press.
Chapelle, C. A., Enright, M. E., & Jamieson, J. (Eds.) (2008). Building a validity argument for the Test of English as a Foreign Language. London: Routledge.
Cheng, L. (2005). Changing Language Teaching through Language Testing: A Washback Study. Cambridge: Cambridge University Press.
Cheng, L., Watanabe, Y., & Curtis A. (Eds.) (2004). Washback in Language Testing: Research Contexts and Methods. Mahwah, NJ: Lawrence Erlbaum.
Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement, 2nd ed. (pp. 443–507). Washington, DC: American Council on Education.
Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
Cureton, E. E. (1951). Validity. In E. F. Lindquist (Ed.), Educational measurement. Washington, DC: American Council on Education.
Davin, K. J., Herazo, J. D., & Sagre, A. (2017). Learning to mediate: Teacher appropriation of dynamic assessment. Language Teaching Research, 21(5), 632-651.
Ebel, R. (1961). Must all tests be valid? American Psychologist, 16, 640–647.
Fulcher, G., & Davidson, F. (2007). Language Testing and Assessment. London & New York: Routledge.
Geranpayeh, A. (2003). A quick review of the English Quick Placement Test. Research Notes Quarterly, 12, 8-10.
Guion, R. (1977). Content validity: The source of my discontent. Applied Psychological Measurement, 1, 1–10.
Hanreddy, J. & Whalley, E. (2008). Mosaic 1: Listening and speaking. Maidenhead, UK: Mc graw Hill.
Hashemi Shahraki, S., Ketabi, S., & Barati, H. (2015). Dynamic assessment in EFL classrooms: Assessing listening comprehension in three proficiency levels. International Journal of Research Studies in Education, 4(3),73- 89
Haywood, H.C., & Lidz, C.S. (2007). Dynamic assessment in practice: Clinical and educational applications. Cambridge: Cambridge University Press.
House, E. R. (1980). Evaluating with validity. Beverly Hills, CA: Sage Publications.
Hughes, A. (1989). Testing for Language Teachers. Cambridge: Cambridge University Press.
Kane, M. (1992). An argument-based approach to validation. Psychological Bulletin, 112, 527–535.
Kane M. (2012). Validating score interpretations and uses. Language Testing. 29(1) 3–17.
Kane, M. (2013). Validating the interpretation and uses of test scores. Journal of Educational Measurement, 50 (1), 1-73.
Kane, M. (2017). Loosening psychometric constraints on educational assessments. Assessment in Education: Principles, Policy & Practice, 24, 447–453.
Kane, M. T. (2021). Articulating a validity argument. In The Routledge handbook of language testing (pp. 32-47). Routledge.
Kane, M. T., and Wools., S. (2019). “Perspectives on the validity of classroom assessments,” in Classroom Assessment and educational measurement. Editors S. M. Brookhart, and J. H. McMillan (New York, NY: Routledge), p. 11–26.
Lantolf, J. P., & Poehner, M. E. (2004). Dynamic assessment of L2 development: bringing the past into the future. Journal of Applied Linguistics, 1(2), 49-72.
Lantolf, J. P., & Poehner. M. E. (2007). Dynamic Assessment. In Encylopedia of Language and Education Volume 7. Language Testing and Assessment (E. Shohamy, Ed., N. Hornberger, Gen. Ed.). Springer Publishing.
Lantolf, J. P., & Poehner, M. E. (2013). The unfairness of equal treatment: objectivity in L2 testing and dynamic assessment, Educational Research and Evaluation: An International Journal on Theory and Practice, 19:2-3, 141-157
Mackey, A., Gass, S. (2005). Second language research methodology and design. Mahwah, NJ: Lawrence Erlbaum Associates.
Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. Braun (Eds.), Test validity (pp. 33–45). Hillsdale, NJ: Lawrence Erlbaum.
Messick, S. A. (1989). Validity. In R. L. Linn (ed.), Educational measurement. 3rd Ed. New York: American Council on Education. 13-103.
Mislevy, R. J. (2009). Validity from the perspective of model-based reasoning. In R. W. Lissitz (Ed.), The concept of validity: Revisions new directions and applications (pp. 83–108). Charlotte, NC: Information Age.
Moss, P. A. (2003). Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and Practice 22 (4) 13-25.
Nunan, D. (2002). Listening in language learning. In J. C. Richards & W. A. Renandya, (Eds.). Methodology in language teaching: An anthology of current practice (pp. 238-241). Cambridge: Cambridge University Press
Petrovsky, A. V. (1985). Studies in psychology: The collective and the Individual. Moscow: Progress.
Poehner, M. E. (2005) Dynamic assessment of advanced L2 learners of French. Ph.D. dissertation. Penn State University.
Poehner, M. E. (2008a). Dynamic assessment. A Vygotskian approach to understanding and promoting L2 development. Berlin, Germany: Springer.
Poehner, M. E. (2008b). Dynamic Assessment and the Problem of Validity in the L2 Classroom. (CALPER Working Paper Series, No. 10). The Pennsylvania State University: Center for Advanced Language Proficiency Education and Research.
Poehner, M.E. (2009). Group dynamic assessment: Mediation for the L2 classroom. TESOL Quarterly, 43, 471–491.
Poehner, M. E. (2011). Validity and interaction in the ZPD: Interpreting learner development through L2 Dynamic assessment. International Journal of Applied Linguistics, 21, 244–26
Sternberg, R. J., & Grigorenko, E. L. (2002). Dynamic Testing. New York: Cambridge University Press.
Stobart, G. (2012). Validity in formative assessment. In J. Gardner (Ed.), Assessment and Learning, 2nd edition. London: Sage.
Teasdale, A., & Leung, C. (2000). Teacher assessment and psychometric theory: A case of paradigm crossing? Language Testing 17 (2), 163–84.
Torrance, H. (1995). Teacher involvement in new approaches to assessment. In H. Torrance (ed.), Evaluating authentic assessment. Buckingham: Open University Press. 44-56.
Tyler, M. D. (2001). Resource consumption as a function of topic knowledge in nonnative and native comprehension. Language Learning, 51(2), 257–280.
Vandergrift, L. (2004). Learning to listen or listening to learn. Annual review of applied linguistics, 24, 3-25.
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, (Eds. and Trans.) Cambridge, MA: Harvard University Press. (Original work published in 1955).
Vygotsky, L. S. (1987). Thinking and speech. In R. W. Rieber & A. S. Carton (eds.), The collected works of L. S. Vygotsky. Vol. 1. Problems of general psychology (pp.39-285). New York: Plenum.
Vygotsky, L. S. (1998). The problem of age. In R. W. Rieber (Ed.), Child psychology, Vol. 5. Collected works of L. S. Vygotsky (pp. 187– 205). New York: Plenum. (Original work published 1932–1934).
Warford, M. K. (2010). The zone of proximal teacher development. Teaching and Teacher Education, 27 (2), 252-258.
