Psychometric Characteristics of a Rating Scale for Assessing Interactional Competence in Paired-Speaking Tasks at Micro-level
محورهای موضوعی : language teaching
میلاد رمضانی
1
(Department of English, Tabriz Branch, Islamic Azad University, Tabriz, Iran)
بیوک بهنام
2
(Department of English, Tabriz Branch, Islamic Azad University, Tabriz, Iran)
سعیده آهنگری
3
(Department of English, Tabriz Branch, Islamic Azad University, Tabriz, Iran)
کلید واژه: psychometric characteristics, rating scale, interactional competence, paired-speaking tasks, micro-level,
چکیده مقاله :
Developing rating scales for assessing interactional performance is demanding since it is a relatively complicated procedure. The present study investigated the psychometric characteristics of the CAP rating scale (Wang, 2015) for assessing interactional competence at micro-level. To this end, 160 Iranian intermediate EFL learners were selected based on their performance on TOEFL iBT test from a language institute in Tabriz. Four interaction tasks were used to elicit students’ performance on interactional competence using the CAP rating scale. Five raters were recruited in the study to assign score to each individual’s performance. The participants were pretested and post-tested at the beginning and the end of the term through the same scale. The Pearson correlations were computed in order to estimate the test-retest reliability indices of the scale. In addition, five separate exploratory factor analysis (EFA) through the varimax rotation method were conducted in order to investigate the underlying constructs of the communication functions individually and as a total. The results revealed that the CAP rating scale enjoys a reasonable reliability indices and the four functions i.e. building argument, developing discussion, offering support, and shaping connection can be appropriate predictors of interactional competence. Some pedagogical and assessment implications are presented as well.
بررسی های مربوط به تعامل در زبان دوم که محوریت اجتماعی دارند، در طول 15 سال گذشته تحقیقات بیشماری را به خود اختصاص داده است. اما طراحی مقیاس های سنجش عملکرد تعاملی دشوار است چرا که فرآیند پیچیده ای می­طلبد. پژوهش حاضر بر آن است تا ویژگی های روان­سنجی مقیاس اندازه گیری کپ (ونگ، 2015) برای سنجش توانش تعاملی در سطح خرد را بررسی نماید. بدین منظور 160 زبان آموز سطح متوسط ایرانی از آموزشگاه زبانی در شهر تبریز بر اساس عملکردشان در آزمون تافل اینترنتی انتخاب شدند. عملکرد تعاملی آنان در چهار وظیفه تعاملی شامل مشاهده تفاوت، تصمیم گیری، تکمیل داستان و بحث آزاد با استفاده از مقیاس سنجش مذکور در ابتدا و انتهای دوره مورد پیش آزمون و پس آزمون قرار گرفت. برای تخمین پایایی در پیش­آزمون و پس­آزمون در 13آیتم 4 عاملی این مقیاس، از شاخص همبستگی پییرسون استفاده گردید. علاوه بر آن، 5 تحلیل عاملی اکتشافی از طریق روش گردشی واریماکس به منظور روایی سنجی 4 عملکرد ارتباطی تعیین کننده توانش تعاملی به صورت انفرادی و یکجا مورد استفاده قرار گرفت. نتایج حاکی از آن بود که مقیاس سنجش کپ از شاخص پایایی قابل قبولی برخوردار است و چهار عملکرد ارتباطی: ارائه استدلال، بسط مباحثات، انجام مساعدت/حمایت و ایجاد ارتباط بین موضوعات می­توانندعامل پیش بینی کننده توانش تعاملی باشند. پیرو این نتایج برخی استنباط های آموزشی و ارزشیابی ارائه شده است.
References
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Brindley, G. (1998). Describing language development: Rating scales and SLA. In L. F. Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 112-140). Cambridge: Cambridge University Press.
Bachman & A. D. Cohen (Eds.), Interface between second language acquisition and language testing research (pp. 112-140). Cambridge: Cambridge University Press.
Brooks, L. (2009). Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26(3), 341–366.
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1-47.
Davis, L. (2009). The influence of interlocutor proficiency in a paired oral assessment. Language Testing, 26(3), 367–396.
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters' orientation to interaction. Language Testing, 26(3), 423–443.
Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.
Filed, A. (2013). Discovering Statistics Using IBM SPSS, Statistics for Statistics. (4th ed.). London: SAGE Publications.
Fulcher, G. (2003). Testing second language speaking: Pearson Education.
Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113–132.
Fulcher, G., Davidson, F., & Kemp, J. (2011). Effective rating scale development for speaking tests: Performance decision trees. Language Testing, 28(1), 5–29.
Galaczi, E. (2004). Peer-peer interaction in a paired speaking test: the case of the First Certificate in English (Unpublished doctoral dissertation). Columbia University, New York.
Galaczi, E. D. (2013). Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics, 35(5), 553–574.
Hall, J. K. (1995). (Re)creating our worlds with words: A sociohistorical perspective of face-toface interaction. Applied Linguistics, 16(2), 206–232.
He, A. W., & Young, R. (1998). Language proficiency interviews: A discourse approach. Talking and testing: Discourse approaches to the assessment of oral proficiency, 14, 1–24.
Hymes, D. H. (1972a). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp. 269–293). Harmondsworth, England: Penguin.
Jacoby, S., & Ochs, E. (1995). Co-construction: An introduction. Research on Language and Social Interaction, 28 (3), 171-183.
Jin, T., Mak, B., & Zhou, P. (2012). Confidence scoring of speaking performance: How does fuzziness become exact? Language Testing, 29(1), 43–65.
Kley, K. (2015). Interactional competence in paired speaking tests: role of paired task and test-taker speaking ability in co-constructed discourse (Unpublished doctoral dissertation). University of Iowa, Iowa.
Kramsch, C. (1986). From language proficiency to interactional competence. The Modern Language Journal, 70(4), 366–372.
Lado, R. (1961). Language Testing: The Construction and Use of Foreign Language Tests. A Teacher's Book. New York: McGraw-Hill Book.
Long, M. H. (1981). Input, interaction, and second‐language acquisition. In H. Winitz (Ed.), Native Language and Foreign Language Acquisition, (pp. 259-278). New York: Annals of the New York academy of sciences.
Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press.
May, L. (2009). Co-constructed interaction in a paired speaking test: The rater's perspective. Language Testing, 26(3), 397–421.
May, L. (2011). Interactional competence in a paired speaking test: Features salient to raters. Language Assessment Quarterly, 8(2), 127–145.
McNamara, T. F. (1997). ‘Interaction’ in second language performance assessment: Whose performance? Applied Linguistics, 18(4), 446–466.
North, B. (1995). The development of a common framework scale of descriptors of language proficiency based on a theory of measurement. System, 23(4), 445-465.
Norton, J. (2005). The paired format in the Cambridge Speaking Tests. ELT Journal, 59(4), 287–297.
Ohta, A. S. (2000). Rethinking interaction in SLA: Developmentally appropriate assistance in the zone of proximal development and the acquisition of L2 grammar. In J. P. Lantolf (Ed), Sociocultural theory and second language learning, (pp. 51–78). Oxford: Oxford University Press.
Oksaar, E. (1990). Language contact and culture contact: Towards an integrative approach in second language acquisition research. Current Trends in European Second Language Acquisition Research. Multilingual Matters, Clevendon, 10–20.
Pallant, J. (2013). SPSS survival manual. London: McGraw-Hill Education.
Poonpon, K. (2009). Expanding a second language speaking rating scale for instructional and assessment purposes. Arizona: Northern Arizona University Press.
Samuda, V., & Bygate, M. (2008). Tasks in second language learning. Basingstoke: Palgrave Macmillan.
Sun, Y. (2012). The influence of the social interactional context on test performance: A sociocultural view. Canadian Journal of Applied Linguistics/Revue canadienne de linguistique appliquée, 14(1), 194–221.
Taylor, L. (2001). The paired speaking test format: Recent studies. University of Cambridge ESOL Examinations Research Notes, 6, 15-17.
Taylor, L., & Wigglesworth, G. (2009). Are two heads better than one? Pair work in L2 assessment contexts: Sage Publications Sage UK: London, England.
Wang, L. (2015). Assessing interactional competence in second language paired speaking tasks: (Unpublished doctoral dissertation). Northern Arizona University, Arizona.
