The Impact of Task Types and Rating Methods on Iranian EFL Learners’ Speaking Scores
Subject Areas : Research in English Language PedagogyShokouh Rashvand Semiyari 1 , Soghra Ahangari 2
1 - Department of English Language Teaching, Islamic Azad University, West Tehran Branch, Tehran-Iran
2 - Department of English Language Teaching, Islamic Azad University, East Tehran Branch, Tehran-Iran
Keywords: speaking scores, holistic and analytic rating methods, task types, Assessment,
Abstract :
Speaking assessment is quite challenging as there are many factors in addition to speakers’ ability contributing to how well someone can speak a language. In this study, 40 male and female upper intermediate EFL learners, with age range of 15-26, selected through ECCE test. They performed four different types of speaking tasks (explaining, problem-solving, story-telling, and picture-describing). These tasks were rated by two raters using two scoring methods: holistic and analytic. The one-way repeated measures ANOVAs, paired-sample t-tests and Pearson Product Moment Correlations illustrated that the task types and rating methods didn’t have any significant effects on learners’ speaking scores as far as problem-solving, explaining, and picture-describing tasks were concerned. Yet, the rating methods represented to have some effects on story-telling task. The findings also indicated a significant correlation between holistic and analytic ratings of the problem-solving, picture-describing and story-telling tasks while the reverse was true for the explaining task.
Airasian, P. W., & Russell, M. K. (2001). Classroom assessment: Concepts and applications. New York: McGraw- Hill.
Branden, K. (2006). Task-based language education: from theory to practice. Library of Congress Cataloging-in-Publication Data, 54(4), 1027-1050.
Branden, K., Gorp, K., & Verhelst, M. (2009). Tasks in action: Task-based language education from a classroom-based perspective. UK: Cambridge Scholars Publishing.
Brindley, G. (1994). Task-centered assessment in language learning programs: the promise and the challenge. In Bird, N., Falvey, P., Tsui, A., Allison, D. and McNeill, A. (eds.), HongKong. Language and learning, 32(2),73-94.
Brindley, G. (1998). Outcomes-based assessment and reporting in language learning programs: a review of the issues. SAGE Journals, 15(1), 45-85.
Bygate, M. (1999). Task as context for the framing, reframing and unframing of language.
System, 27(1), 33-48.
Bygate, M., Skehan, P. & Swain, M. (2001). Effects of task repetition on the structure and
control of oral language. Researching pedagogic tasks: Second language learning, teaching and testing, 27(11), 23-48.
Candlin, C.N. (1987). Towards task-based language learning. In Candlin, C.N. & Murphy, D.F. (eds.). Language Learning Tasks, 15(6), 5-22.
Chi, E. (2001). Comparing holistic and analytic scoring for performance assessment with many-facet Rasch model. Journal of applied measurement, 2(4), 379-388.
Colomar, A. M. P. (2014). A classroom-based assessment method to test speaking skills in English for Specific Purposes. Language Learning in Higher Education, 4(1), 9-26.
Eckes, T. (2005). Examining Rater Effects in TestDaf Writing and Speaking Performance Assessments: A Many Facet Rasch Analysis, Language Assessment Quarterly, 2 (3), 197-221.
Ellis, N. C. (2001). Memory for language. In Robinson, P. (ed.), Cognition and Second Language Instruction. Cambridge: Cambridge University Press.
Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.
Ellis, R. (2006). The methodology of task-based teaching. Asian EFL journal, 8(3), 19-45.
Ghalib, T. K., & Al-Hattami, A. A. (2015). Holistic versus Analytic Evaluation of EFL Writing: A Case Study. English Language Teaching, 8(7), 225.
Han, T. & Huang, J. (2017). Examining the impact of scoring methods on the institutional EFL writing assessment: A Turkish Perspective. Journal of Language Teaching and Learning, 53, 112-147.
Harsch, C., & Martin, G. (2013). Comparing holistic and analytic scoring methods: Issues of validity and reliability. Assessment in Education: Principles, Policy & Practice, 20 (3), 281- 307.
Huang, H. C. (2007). Lexical context effects on speech perception in Chinese people with autistic traits. Journal of child language, 27 (1), 3-42.
Hughes, J. (1989). Why functional programming matters. The computer journal, 32 (2), 98-107.
Hunter, D. M., Jones, R. M., & Randhawa, B. S. (1996). The use of holistic versus analytic scoring for large-scale assessment of writing. The Canadian Journal of Program Evaluation, 11(2), 61.
In’nami, Y. & Koizumi, R. (2015). Task and rater effects in L2 speaking and writing: A synthesis of generalizability studies. Language Testing, 33 (3), 341-366.
Jackson, S. (2011). Social works: Performing art, supporting publics. Routledge: Routledge Publications.
Kim, Y.J., Payant, C., & Pearson, P. (2015). The Intersection of Task-Based Interaction, Task Complexity, and Working Memory: L2 Question Development through Recasts in a Laboratory Setting. Studies in Second Language Acquisition, 37 (3), 549-581.
Knoch, U., Macqueen, S., & O'Hagan, S. (2014). An investigation of the effect of task type on the discourse produced by students at various score levels in the TOEFL iBT® writing test. ETS Research Report Series, 2014 (2), 1-74.
Lee, Y. W. (2006). Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks. Language Testing, 23(2), 131-166.
Lim, C., & Lee, J.H. (2015).The effect of task modality and type on Korean EFL learners’ Interactions. Journal of Asia TEFL, 12 (2), 87-13.
Long, M. (2015). Second language acquisition and task-based language teaching. Applied Linguistics, 37 (3),438-441.
Lumley, T., & O’Sullivan, B. (2005). The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speaking. Language Testing, 22(4), 415-437.
Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press.
McCutchen, D. (1986). Domain knowledge and linguistic knowledge in the development of writing ability. Journal of Memory and Language, 25, 431-444.
McGraw, H. (2015). 6 types of assessment of learning.Teach Thought Staff, 24, 517.
Park, Y., & Park, G. (2004). A new method for technology evaluation in monetary value: procedure and application. Technovation, 24(5), 387-394.
Pica, T., Kanagy, R., & Falodun, J. (1993). Choosing and using communication tasks for second language instruction. Multilingual Matters, 9-9.
Prabhu, N. S. (1987). Second Language Pedagogy. Oxford: Oxford University Press.
Reinders, H. (2009). Study Skills for Speakers of English as a Second Language. Genome Research, 18(3), 469-76.
Richards, J. C., & Rodgers, T. S. (2014). Approaches and methods in language teaching.
Cambridge: Cambridge university press.
Saeidi, M., & Rashvand, Sh. (2012). The Impact of Rating Methods and Task Types on EFL Learners' Writing Scores. Journal of English Studies, 1, 59-68.
Samuda, V. (2001). Guiding relationships between form and meaning during task performance: The role of the teacher. Researching pedagogic tasks: Second language learning, teaching and testing, 8 (3), 119-140.
Salvia, J., Ysseldyke, J. E., & Bolt, S. (2007). Assessment in special and inclusive education. US: Cengage Learning.
Sawaki, Y. (2007). Construct validation of analytic rating scales in a speaking assessment:
Reporting a score profile and a composite. Language Testing, 24(3), 355-390.
Skehan, P. (1998). Task-based instruction. Annual review of applied linguistics, 18, 268-286.
Smith, j. (2003). Reconsidering Reliability in Classroom Assessment and Grading. Educational Measurement Issues and Practice, 22(4), 26-33.
Swain, M. (1995). Three functions of output in second language learning. Principle & practice in applied linguistics, Oxford: Oxford University Press.
Teng, H. C. (2007). A Study of Task Types for L2 Speaking Assessment. Retrieved August 25, 2017 from WWW. eric.ed.gov.
Teng, H. & Huei-Chun, S. (2007). A study of Task Types for L2 Speaking Assessment, Institute of Education Sciences, 11.
Tuan, L. T. (2012). Teaching and assessing speaking performance through analytic scoring
approach. Theory and Practice in Language Studies, 2(4), 673.
Upshur, J.A. & Turner, C.E. (1999). Systematic effects in the rating of second-language speaking ability: test method and learner discourse. Language Testing, 16 (1), 82-111.
Willis, D., and Willis, J. (2007(. Doing Task-based Teaching. Oxford: Oxford University Press.
Wiseman, C. S. (2012). A comparison of the performance of analytic vs. holistic scoring rubrics to assess L2 writing. Iranian Journal of Language Testing, 2 (1), 59-92.