A Cross-Cultural Exploration: Developing an MDCT for Pragmatic Competence in EFL and Native English Speakers, with Emphasis on Speech Acts of Request and Apology
محورهای موضوعی : Research in English Language Pedagogy
Zahra Jafari
1
,
Hamid Dowlatabadi
2
1 - English Language and Literature Department, Faculty of Literature and Languages, Arak University, Arak, Iran
2 - English Language and Literature Department, Faculty of Literature and Languages, Arak University, Arak, Iran
کلید واژه: Apology, Cross-cultural exploration, Discourse Completion Test, MDCT (Multiple Discourse Completion Test), Pragmatic Competence, Request, Speech Act,
چکیده مقاله :
Pragmatic competence, the ability to perform language functions appropriately across social and cultural contexts, is a critical component of effective intercultural communication. However, existing assessment tools often lack contextual grounding and cultural sensitivity, particularly in evaluating speech acts such as requests and apologies. This study aims to develop a culturally grounded and empirically validated Multiple-choice Discourse Completion Test (MDCT) to assess pragmatic competence among EFL learners. Drawing on data from 147 participants (21 native English speakers and 126 Iranian and Iraqi EFL learners), the research employed an eight-phase design, which included exemplar generation, likelihood investigation, metapragmatic assessment, pilot testing, expert review, item analysis, scoring rubric development, and statistical validation. Data were collected through structured questionnaires, Likert-scale evaluations, expert judgments, and learner feedback. Quantitative analyses confirmed the test’s internal consistency as well as construct validity, and criterion-related validity. Qualitative procedures ensured the cultural plausibility and contextual appropriateness of items. The final MDCT demonstrated high inter-rater reliability and effective discrimination across pragmatic proficiency levels. By embedding native speaker input from the beginning, the instrument addresses cultural bias often found in traditional pragmatic assessments. The findings contribute to ongoing efforts to design fair, reliable, and pedagogically relevant tools for assessing intercultural pragmatic competence. This work offers both theoretical insight into the multi-dimensional nature of pragmatics and practical applications for EFL instruction and curriculum development.
Pragmatic competence, the ability to perform language functions appropriately across social and cultural contexts, is a critical component of effective intercultural communication. However, existing assessment tools often lack contextual grounding and cultural sensitivity, particularly in evaluating speech acts such as requests and apologies. This study aims to develop a culturally grounded and empirically validated Multiple-choice Discourse Completion Test (MDCT) to assess pragmatic competence among EFL learners. Drawing on data from 147 participants (21 native English speakers and 126 Iranian and Iraqi EFL learners), the research employed an eight-phase design, which included exemplar generation, likelihood investigation, metapragmatic assessment, pilot testing, expert review, item analysis, scoring rubric development, and statistical validation. Data were collected through structured questionnaires, Likert-scale evaluations, expert judgments, and learner feedback. Quantitative analyses confirmed the test’s internal consistency as well as construct validity, and criterion-related validity. Qualitative procedures ensured the cultural plausibility and contextual appropriateness of items. The final MDCT demonstrated high inter-rater reliability and effective discrimination across pragmatic proficiency levels. By embedding native speaker input from the beginning, the instrument addresses cultural bias often found in traditional pragmatic assessments. The findings contribute to ongoing efforts to design fair, reliable, and pedagogically relevant tools for assessing intercultural pragmatic competence. This work offers both theoretical insight into the multi-dimensional nature of pragmatics and practical applications for EFL instruction and curriculum development.
Ahn, R. (2005). Self-assessment, language lab oral production tests, open discourse completion tests, role-play, role-play self-assessment, and multiple-choice discourse completion tests (MDCT) for cross-cultural pragmatics. Language Testing, 22(4), 393–419.
Alemi, M., & Khanlarzadeh, N. (2016). Pragmatic assessment of request speech act of Iranian EFL learners by non-native English-speaking teachers. Iranian Journal of Language Teaching Research, 4(2), 19–34.
Arabmofrad, A., & Mehdiabadi, F. (2022). Developing a multiple-choice discourse completion test for Iranian EFL learners: The case of the four speech acts of apology, request, refusal, and thanks. Language Related Research, 13(4), 1–26.
Austin, J. L. (1962). How to do things with words. Oxford University Press.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
Bardovi-Harlig, K., & Dörnyei, Z. (1998). Do language learners recognize pragmatic violations? Pragmatic vs. grammatical awareness in instructed L2 learning. TESOL Quarterly, 32(2), 233–259.
Birjandi, P., & Rezaei, S. (2010). Developing a multiple-choice discourse completion test of interlanguage pragmatics for Iranian EFL learners. ILI Language Teaching Journal, 6(1), 2–17.
Bouton, L. F. (1999). Developing non-native speakers' knowledge of implicature in English: Acculturating to the expected use of routines. In E. Hinkel (Ed.), Culture in second language teaching and learning (pp. 47–70). Cambridge University Press.
Brown, J. D. (2001). Testing in language programs. Prentice Hall Regents.
Chalak, A. (2021). Pragmatic competence in digital discourse: Self-presentation and praise in Iranian EFL social media users. Journal of Intercultural Communication Research, 50(2), 145–162.
Cohen, A. D. (2008). Speaking, listening and writing. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 177–196). Routledge.
Derakhshan, A., & Eslami, Z. R. (2020). The role of teachers in developing pragmatic competence: A case of Iranian EFL context. Language and Intercultural Communication, 20(5), 552–567.
Derakhshan, A., Shakki, F., & Malmir, A. (2021). Pragmatic assessment in EFL contexts: From theory to practice. Applied Pragmatics, 3(1), 28–46.
Douglas, D., & Myers, B. (2022). Fairness and justice in language testing. Language Assessment Quarterly, 19(3), 217–237.
Farhady, H. (1980). Systematic development of a pragmatic competence test based on a functional approach. Language Testing, 7(2), 155–170.
García, M. (2020). Cultural scripts and pragmatic assessment in EFL contexts. Journal of Intercultural Language Learning, 12(3), 123–138.
García, M. (2020). Cultural scripts and their application in pragmatic testing: Toward more inclusive assessment practices. Journal of Applied Linguistics, 42(3), 245–263. https://doi.org/10.1016/j.jal.2020.04.005
Garcia, R., & Martinez, L. (2022). Context sensitivity and cultural awareness in pragmatic assessment. Journal of Applied Linguistics, 12(2), 203–226.
Glaser, K. (2022). Cultural bias in DCT-based pragmatic tests: A re-evaluation. Intercultural Pragmatics, 19(1), 35–58.
Greenfield, T. (2021). Validating classroom-based pragmatic assessments. Language Testing in Asia, 11(1), 1–15.
Holmes, J., & Brown, D. F. (1987). Effectiveness of teaching pragmatics and the developmental process of learners' pragmatic competence. Applied Linguistics, 8(2), 218–235.
Hudson, T., Detmer, E., & Brown, J. D. (1992). Major exploration of interlanguage pragmatic knowledge: Indirect, semi-direct, and self-assessment measures. Studies in Second Language Acquisition, 14(3), 305–324.
Hudson, T., Detmer, E., & Brown, J. D. (1995). Developing prototypic measures of cross-cultural pragmatics. University of Hawai’i Press.
Hudson, T., Detmer, E., & Brown, J. D. (1995). Exploration of interlanguage pragmatic knowledge with multiple test formats. Language Testing, 12(3), 305–322.
Ishihara, N. (2020). Incorporating pragmatics into L2 assessment: New trends. Journal of Pragmatic Studies, 7(2), 113–129.
Ishihara, N., & Cohen, A. D. (2010). Teaching and learning pragmatics: Where language and culture meet. Pearson Education.
Ishihara, N., & Nguyen, D. (2021). Teaching and assessing EIL pragmatics: A context-sensitive approach. Language and Intercultural Communication, 21(4), 487–503. https://doi.org/10.1080/14708477.2021.1888743
Ishihara, N., & Nguyen, L. T. (2021). Reconceptualizing pragmatics instruction for English as an international language. Language Teaching Research, 25(5), 671–690. https://doi.org/10.1177/1362168820921896
Jianda, L. (2006a). Measuring interlanguage pragmatic knowledge of EFL learners. Peter Lang.
Jianda, L. (2006b). Assessing EFL learners’ interlanguage pragmatic knowledge: Implications for testers and teachers. Reflections on English Language Teaching, 5(1), 1–22.
Jianda, L. (2007). Developing a pragmatics test for Chinese EFL learners. Language Testing, 24(3), 391–415.
Kasper, G., & Dahl, M. (1991). Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 13(2), 215–247.
Kasper, G., & Liu, F. (2020). Dynamic assessment in interlanguage pragmatics. Applied Pragmatics, 2(1), 1–21. https://doi.org/10.1075/ap.00010.kas
Kasper, G., & Liu, J. (2020). Pragmatic development and assessment in second language learning. Annual Review of Applied Linguistics, 40, 33–51. https://doi.org/10.1017/S0267190519000238
Kasper, G., & Rose, K. R. (2002). Pragmatic development in a second language. Blackwell.
Kasper, G., & Rose, K. R. (2002). Suitable instruments for eliciting interlanguage pragmatics knowledge. In K. R. Rose & G. Kasper (Eds.), Pragmatics in language teaching (pp. 125–144). Cambridge University Press.
Khatib, M., & Rezaei, S. (2023). A systematic review of pragmatic assessment in EFL contexts: Challenges and directions. Pragmatics and Language Learning, 31(1), 55–77. https://doi.org/10.1016/j.praglang.2023.02.003
Lee, J., Kang, S., & Hyun, J. (2020). Context sensitivity and cultural awareness in pragmatic assessment. Journal of Pragmatics, 167, 1–15.
Li, S., & Taguchi, N. (2023). Measuring L2 pragmatic competence: A systematic review. Language Teaching Research, 27(1), 31–50.
Linacre, J. M. (2012). Winsteps Rasch measurement computer program. https://www.winsteps.com/
Liu, J. (2007). Pragmatic test for assessing Chinese EFL students' pragmatic knowledge in the context of an apology speech act. Language Testing, 24(3), 391–415. https://doi.org/10.1177/0265532207077206journals.sagepub.com
Liu, Y. (2023). The role of metapragmatic instruction in speech act performance: Evidence from EFL learners. Pragmatics and Language Learning, 20(1), 78–97.
Lumley, T. (2005). Assessing second language writing: The rater’s perspective. Peter Lang.
Ma, L., Zhang, Q., & Thompson, J. (2025). Evaluating large language models’ capacity for pragmatic reasoning in EFL settings. Journal of Educational NLP, 7(2), 56–73.
Ma, Z., Yuan, Y., & Li, C. (2025). Assessing large language models on pragmatic inference: Implicature, politeness, and indirectness. Computational Linguistics Review, 47(2), 99–124. https://doi.org/10.1145/xxxxxxx
McNamara, T. (2000). Language testing. Oxford University Press.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Paluanova, E. (2024). Integrating traditional and digital tools in pragmatic assessment: A comparative study. Language Assessment Quarterly, 21(1), 45–68. https://doi.org/10.1080/15434303.2024.1990012
Purpura, J. E. (2004). Assessing grammar. Cambridge University Press.
Razmjoo, S. A. (2007). Cultural patterns in Persian EFL learners' compliment responses: Implications for interlanguage pragmatics. International Journal of Language Studies, 1(3), 45–62.
Roever, C. (2005). Inclusion of speech acts, implicature, and routine formulas in pragmatic tests. Journal of Pragmatics, 37(10), 1645–1669. https://doi.org/10.1016/j.pragma.2004.12.001
Roever, C. (2005). Testing ESL pragmatics: Development and validation of a web-based assessment battery. Peter Lang.
Roever, C. (2021). Rethinking pragmatic tests: Moving beyond DCTs. Language Testing, 38(3), 423–441. https://doi.org/10.1177/0265532221998325
Rose, K. R., & Kasper, G. (2001). Pragmatics in language teaching. Cambridge University Press.
Salehi, M., & Isavi, E. (2013). Developing a test of interlanguage pragmatics for Iranian EFL learners in relation to the speech acts of request and apology. Iranian Journal of Language Studies, 1(1), 1–16.
Searle, J. R. (1969). Speech acts: An essay in the philosophy of language. Cambridge University Press.
Schauer, G. (2021). Revisiting pragmatic assessment in interlanguage research. Pragmatics and Society, 12(3), 317–340.
Smith, P., & Johnson, M. (2021). Recent perspectives on context sensitivity in pragmatic assessment. Applied Linguistics Review, 12(2), 203–226. https://doi.org/10.1515/applirev-2020-0010
Taguchi, N. (2012). Context, individual differences and pragmatic competence. Multilingual Matters.
Taguchi, N. (2022). Toward a socio-cognitive model of pragmatic assessment. Journal of Pragmatics, 195, 101–113. https://doi.org/10.1016/j.pragma.2022.04.002
Taguchi, N., & Roever, C. (2021). Second Language Pragmatics. Oxford University Press.
Tajeddin, Z., Alemi, M., & Pashmforoosh, R. (2022). Pragmatic assessment in SLA research: Challenges and innovations. Studies in Second Language Learning and Teaching, 12(1), 45–70.
Timpe-Laughlin, V. (2020). Language assessment literacy for a multilingual era. Language Testing, 37(4), 541–549.
Verschueren, J. (2012). Contextualizing pragmatic competence. John Benjamins Publishing.
Wang, Q., & Spencer-Oatey, H. (2022). Intercultural pragmatics assessment: Limitations and future directions. Intercultural Pragmatics, 19(4), 563–585. https://doi.org/10.1515/ip-2022-0025
Yeshanov, I. (2025). Task-based pragmatic assessment in EFL: Addressing learners' communicative needs. TESOL Quarterly, 59(1), 75–98. https://doi.org/10.1002/tesq.3456
