Diagnostic Assessment of Interactional Competence in Paired Speaking Tests: Investigating Rating Accuracy of Iranian EFL Learners

Subject Areas : All areas of language and translation

Masoome Azmoode Sis Abad ¹ , Gholam-Reza Kiyani ² , Gholam-Reza Abbasian ³

1 - Department of English, Islamic Azad University, Science and Research Branch, Tehran, Iran
2 - Department of English, Tarbiat Modares University, Tehran, Iran.
3 - Department of English, Imam Ali University, Tehran, Iran

Received: 2022-04-11 Accepted : 2023-07-29 Published : 2024-01-01

Keywords: interactional competence, Diagnostic assessment, Assessment stakeholders, Paired speaking tests,

Abstract :

Assessment of interactional competence (IC) as a multi-componential construct poses several challenges for both teachers and administrators. Having a diagnostic assessment perspective through stakeholders’ involvement, this mixed methods study attempts to provide information about students’ strengths and weaknesses in IC. The paper first explored the distinctive effects of diagnostic self- and peer-assessments on the development of Iranian EFL learners’ IC; it then examined the accuracy of the learners’ diagnostic assessment of IC in paired speaking tests. The learners’ perception toward the application of diagnostic assessment was also investigated qualitatively. To this end, 60 students majoring in English translation at Islamic Azad University participated in this study. Taking the instructor’s ratings as the criterion, over the course of 12 weeks, the accuracy of the learners’ diagnostic self- and peer- assessments was investigated. Data analysis, using T-test and MANOVA, confirmed that while the two groups of diagnostic self- and peer- assessments had considerable improvement in IC, there was not any significant difference between the two groups’ gain. In addition, no statistically significant difference was found between the accuracy of diagnostic self-, peer – assessment and those of instructor- assessment throughout the course. Furthermore, the results showed that the participants held favorable perception toward the application of diagnostic assessment.

References:

Alderson, J. C. (2005). Diagnosing Foreign Language Proficiency: The Interface Between Learning and Assessment (illustrated, reprint ed.). Bloomsbury Academic.
Alderson, J. C., Brunfaut, T., & Harding, L. (2014). Towards a Theory of Diagnosis in Second and Foreign Language Assessment: Insights from Professional Practice Across Diverse Fields. Applied Linguistics, 36(2), 236-260.
Borger, L. (2019). Assessing Interactional Skills in A Paired Speaking Test: Raters’ Interpretation of The Construct. Apples - Journal of Applied Language Studies, 13(1), 151-174.
Canale, M., & Swain, M. (1980). Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing. Applied Linguistics, I(1), 1-47.
Chen, Y. M. (2008). Learning to Self-Assess Oral Performance In English: A Longitudinal Case Study. Language Teaching Research, 12(2), 235-262.
Cheng, W., & Warren, M. (2005). Peer Assessment of Language Proficiency. Language Testing, 22(1), 93-121.
Chomsky, N. (1965). Aspects of the Theory of Syntax (Vol. 11). MIT press.
De Grez, L., Valcke, M., & Roozen, I. (2012). How Effective Are Self- And Peer Assessment of Oral Presentation Skills Compared with Teachers’ Assessments? Active Learning in Higher Education, 13(2), 129-142.
Dings, A. (2014). Interactional Competence and the Development of Alignment Activity. The Modern Language Journal, 98(3), 742-756.
Falchikov, N., & Boud, D. (1989). Student Self-Assessment in Higher Education: A Meta-Analysis. Review of Educational Research, 59(4), 395-430.
Galaczi, E., & Taylor, L. (2018). Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, 15(3), 219-236.
Han, C. (2018). A Longitudinal Quantitative Investigation Into The Concurrent Validity of Self and Peer Assessment Applied To English-Chinese Bi-Directional Interpretation in An Undergraduate Interpreting Course. Studies in Educational Evaluation, 58, 187-196.
Han, C., & Fan, Q. (2020). Using Self-Assessment As A Formative Assessment Tool in An English-Chinese Interpreting Course: Student Views and Perceptions of Its Utility. Perspectives, 28(1), 109-125.
Han, C., & Riazi, M. (2018). The Accuracy of Student Self-Assessments of English-Chinese Bidirectional Interpretation: A Longitudinal Quantitative Study. Assessment & Evaluation in Higher Education, 43(3), 386-398.
Hirai, A., & Yokouchi, Y. (2019). An Investigation of EFL Learners' Diagnostic Assessment Capabilities for a Classroom-Based Speaking Test. ARELE: Annual Review of English Language Education in Japan, 30, 209-224.
Jang, E. E. (2005). A Validity Narrative: Effects of Reading Skills Diagnosis on Teaching and Learning in The Context of NG TOEFL. University of Illinois at Urbana-Champaign.
Jang, E. E., Dunlop, M., Park, G., & van der Boom, E. H. (2015). How Do Young Students With Different Profiles of Reading Skill Mastery, Perceived Ability, and Goal Orientation Respond to Holistic Diagnostic Feedback? Language Testing, 32(3), 359-383.
Kazemi, N., & Tavassoli, K. (2020). The Comparative Effect of Dynamic vs. Diagnostic Assessment on EFL Learners’ Speaking ability. Research in English Language Pedagogy, 8(2), 223-241.
Kramsch, C. (1986). From Language Proficiency to Interactional Competence. The Modern Language Journal, 70(4), 366-372.
Lam, D. M. K. (2018). What Counts As “Responding”? Contingency on Previous Speaker Contribution As A Feature Of Interactional Competence. Language Testing, 35(3), 377-401.
Lam, D. M. K. (2019). Interactional Competence with and without Extended Planning Time in a Group Oral Assessment. Language Assessment Quarterly, 16(1), 1-20.
Lee, S.-K., & Chang, S.-H. (2005). Lerner Involvement in Self- and Peer- assessment of Task-based Oral Performance. Second Language Research, 41, 711-735.
Lee, Y.-W. (2015). Diagnosing Diagnostic Language Assessment. Language Testing, 32(3), 299-316.
Lee, Y.-W., & Sawaki, Y. (2009). Application of Three Cognitive Diagnosis Models to ESL Reading and Listening Assessments. Language Assessment Quarterly, 6(3), 239-263.
Lu, L. (2018). An Analysis of Peer-Assessment in Chinese as a Second Language Classroom Presentation. Chinese Language Teaching Methodology and Technology, 1(3), 18.
Ma, W., & Winke, P. (2019). Self-Assessment: How Reliable Is It in Assessing Oral Proficiency Over Time? Foreign Language Annals, 52(1), 66-86.
May, L. (2009). Co-constructed interaction in a paired speaking test: The rater's perspective. Language Testing, 26(3), 397-421.
May, L. (2011). Interactional Competence in a Paired Speaking Test: Features Salient to Raters. Language Assessment Quarterly, 8(2), 127-145.
May, L., Nakatsuhara, F., Lam, D., & Galaczi, E. (2020). Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language Testing, 37(2), 165-188.
Nakatsuhara, F. (2014). The Co-construction of Conversation in Group Oral Tests. Peter Lang Verlag. https://www.peterlang.com/document/1044236
Nakatsuhara, F., May, L., Lam, D., & Galaczi, E. (2018). Learning Oriented Feedback in The Development and Assessment of Interactional Competence (Research Notes, Issue 70). Cambridge Assessment English.
Oscarson, M. (1989). Self-Assessment of Language Proficiency: Rationale and Applications. Language Testing, 6(1), 1-13.
Poehner, M. E., Zhang, J., & Lu, X. (2015). Computerized Dynamic Assessment (C-Da): Diagnosing L2 Development According to Learner Responsiveness to Mediation. Language Testing, 32(3), 337-357.
Roever, C., & Kasper, G. (2018). Speaking in Turns and Sequences: Interactional Competence as A Target Construct in Testing Speaking. Language Testing, 35(3), 331-355.
Ross, S. (2018). Listener Response as A Facet of Interactional Competence. Language Testing, 35(3), 357-375.
Salehi, M., & Masoule, Z. S. (2017). An Investigation of The Reliability And Validity of Peer, Self-, and Teacher Assessment. Southern African Linguistics and Applied Language Studies, 35(1), 1-15.
Skogmyr Marian, K., & Balaman, U. (2018). Second Language Interactional Competence and Its Development: An Overview of Conversation Analytic Research on Interactional Change Over Time. Language and Linguistics Compass, 12(8), e12285.
Tabachnick, B. G., Fidell, L. S., & Ullman, J. B. (2007). Using Multivariate Statistics (Vol. 5). pearson Boston, MA.
Tecedor, M. (2016). Beginning Learners' Development of Interactional Competence: Alignment Activity. Foreign Language Annals, 49(1), 23-41.
Terwase, T. N., & Oluwatoyin, C. (2014). Diagnostic Assessment: A Tool for Quality Control in Education. Educ Res Rev, 1(1), 17-24.
Thomas, G., Martin, D., & Pleasants, K. (2011). Using Self-and Peer-assessment to Enhance Students’ Future-learning in Higher Education. Journal of University Teaching & Learning Practice, 8(1), 52-69.
Tozcu, A. (2016). The Effectiveness of Diagnostic Assessment on the Development of Turkish Language Learners’ Narrative Skills as an Oral Proficiency Interview (OPI) Task. Journal of the National Council of Less Commonly Taught Languages, 19, 61-96.
Youn, S. J. (2020). Managing Proposal Sequences in Role-Play Assessment: Validity Evidence of Interactional Competence Across Levels. Language Testing, 37(1), 76-106.
Young, R. F. (2000). Interactional Competence: Challenges for Validity Annual Meeting of the American Association of Applied Linguistics, Vancouver, Canada.
Young, R. F. (2013). Learning to Talk The Talk and Walk The Walk: Interactional Competence in Academic Spoken English. Ibérica, 25, 15-38.