Testing the visible
literate biases in oral language testing
DOI:
https://doi.org/10.1558/japl.v1.i3.295Keywords:
spoken language, language testing, oral assessment, test paradignms, test validity, test construction, spoken versus written languageAbstract
This paper argues that test criteria for oral assessment tend to be biased towards features more easily captured by written performance than spoken. This argument is carried forward at different levels reflecting the typical areas articulated in evaluation criteria for tests of speaking: discourse management; fluency; lexis and grammar. The paper draws on insights emerging from corpus linguistics, speech processing and sociophonetic studies to suggest that key features of oral communication may be undervalued in traditional approaches to oral assessment. Conversely, linguistic features which are more typical of written mode (for example, wide lexical range) and which are difficult for the native speaker to produce in spontaneous speech are valued positively in second language tests and regarded as key indicators of proficient speaking ability.
References
Biber, D. (1988) Variation across Speech and Writing. Cambridge: Cambridge University Press.
Bonk W. J. and Ockey G. J. (2003) A many-facet Rasch analysis of the second language group oral discussion task. Language Testing 20(1): 89–110.
Boulet J. R., van Zanten M., McKinley D. W. and Gary, N. E. (2001) Evaluating the spoken English proficiency of graduates of foreign medical schools. Medical Education 35(8): 767–773.
Brown, A. (2003) Interviewer variation and the co-construction of speaking proficiency. Language Testing 20(1): 1–25.
Bygate, M. (1999) Quality of language and purpose of task: patterns of learners’ language on two oral communication tasks. Language Teaching Research 3(3): 185–214.
Chafe, W. (1982) Integration and involvement in speaking, writing and oral literature. In D. Tannen (ed.) Spoken and Written Language: exploring orality and literacy. Norwood, NJ: Ablex.
Cunnigham-Andersson, U. (1997) Native speaker reactions to non-native speech. In A. James and J. Leather (eds) Second Language Speech: structure and process 133–44. (Studies on Language Acquisition 13). Berlin and New York: Mouton de Gruyter.
Douglas, D. (2001) Performance consistency in second language acquisition and language testing research: a conceptual gap. Second Language Research 17(4): 442–65.
Foster, P., Tonkyn, A. and Wigglesworth, G. (2000) Measuring spoken language: a unit for all reasons. Applied Linguistics 21(3): 354–75.
Fulcher, G. and Márquez Reiter, R. (2003) Task difficulty in speaking tests. Language Testing 20(3): 321–44.
Gaskell, M. G. and Marslen-Wilson, W. D. (1999) Ambiguity, competition and blending in spoken word recognition. Cognitive Science 23(4): 439–62.
Goody, J. (1977) The Domestication of the Savage Mind. Cambridge: Cambridge University Press.
Goody, J. (1987) The Interface between the Written and the Oral. Cambridge: Cambridge University Press.
Goody, J. (2000) The Power of the Written Tradition. Washington, DC: Smithsonian Institute.
Halliday, M. A. K. (1989) Spoken and Written Language. Oxford: Oxford University Press.
Hughes, R. (1996) English in Speech and Writing: investigating language and literature. London: Routledge.
Hughes, R. (2002) Teaching and Researching Speaking. London: Longman-Pearson.
Hughes, R. (forthcoming a) Assessing speaking: language proficiency versus communicative ability. Proceedings of the Second International Conference on Speech, Writing and Context. Kansai Gaidai: Kansai Gaidai Press.
Hughes, R. (ed.) (forthcoming b) Spoken Language, Applied Linguistics and TESOL: challenges for theory and practice. London: Palgrave-Macmillan.
IELTS (2001) IELTS: International English Language Testing System Handbook (July).
Johnson, M. and Tyler A. (1998) Re-analysing the OPI: how much does it look like natural conversation. In R. Young and A. Weiyn He (eds) Talking and Testing: discourse approaches to the assessment of oral proficiency 27–51. (Studies in Bilingualism 14) Amsterdam and Philadelphia: John Benjamins Publishing Company.
Lazaraton, A. (1992) The structural organisation of a language interview: a conversation analytic perspective. System 20: 373–86.
Leech, G. (2000) Grammars of spoken English, new outcomes of corpus-oriented research. Language Learning 50(4): 675–724.
Lindemann, S. (2002) Listening with an attitude: a model of native-speaker comprehension of non-native speakers in the United States. Language in Society 31(3): 419–41.
Lindemann, S. (2003) Koreans, Chinese, or Indians? Attitudes and ideologies about nonnative English speakers in the United States. Journal of Sociolinguistics 7(3): 348–64.
Lindemann, S. and Mauranen, A. (2001) ‘It’s just real messy’: the occurrence and function of just in a corpus of academic speech. English for Specific Purposes 20: 459–75.
Lu, Y. (2003) Insights into the FCE speaking test. Retrieved from www.cambridgeESOL. org/rs_notes
Luk, J. and Lin, A. (forthcoming) Uncovering the sociopolitical situatedness of accents in English as a second/foreign language (ESL/EFL) speaking assessment. In R. Hughes (ed.) Spoken Language, Applied Linguistics and TESOL: challenges for theory and practice. London: Palgrave-Macmillan.
Lumley, T. (1998) Perceptions of language-trained raters and occupational experts in a test of occupational English language proficiency. English for Specific Purposes 17(4): 347–67.
Mackenzie, C. (2000) Adult spoken discourse: the influences of age and education. International Journal of Language and Communication Disorders 35(2): 269–85.
Malvern, D. and Richards, B. (2002) Investigating accommodation in language proficiency interviews using a new measure of lexical density. Language Testing 19(1): 85–104.
Marslen-Wilson, W. (2003) Speech and language. Foresight Cognitive Systems Project. (Research review) Available at http://www.foresight.gov.uk/
Myford, C. and Wolfe, E. (2000) Monitoring sources of variability within the Test of Spoken English Assessment System. TOEFL Research Report Series. Princeton, New Jersey: Educational Testing Service.
O’Loughlin, K. (2002) The impact of gender in oral proficiency testing. Language Testing 19(2): 169–92.
Olsen, D. R. (1977) From utterance to text: the bias of language in speech and writing. Harvard Educational Review 47(3): 257–81.
Orr, M. (2002) The FCE speaking test: using rater reports to help interpret test scores. System 30(2): 143–54.
O’Sullivan, B. (2000) Exploring gender and oral proficiency interview performance. System 28(3): 373–86.
Riggenbach, H. (1998) Evaluating learner interactional skills: conversation at the micro level. In R. Young and A. Weiyun He (eds) Talking and Testing: discourse approaches to the assessment of oral proficiency 53–67. (Studies in Bilingualism 14) Amsterdam and Philadelphia: John Benjamins Publishing Company.
Rodekohr, R. K. and Haynes, W. O. (2001) Differentiating dialect from disorder: a comparison of two processing tasks and a standardized language test. Journal of Communication Disorders 34: 255–72.
Upshur, J. A. and Turner, C. E. (1999) Systematic effects in the rating of second-language speaking ability: test method and learner discourse. Language Testing 16(1): 82–116.
Verhoven, L., Aparici, M., Cahana-Amitay, D., van Hell, J. G., Kirz, S. and Viguié-Simon, A. (2002) Clause packaging in writing and speech: a cross-linguistic developmental analysis. In R. A. Berman and L. Verhoeven (eds) Cross-Linguistic Perspectives on the Development of Text-Production Abilities in Speech and Writing. Part 2. Special issue of Written Language & Literacy 5(2): 135–61.
Wennerstrom, A. (2001) Intonation and evaluation in oral narratives. Journal of Pragmatics. 33: 1183–206.
Bonk W. J. and Ockey G. J. (2003) A many-facet Rasch analysis of the second language group oral discussion task. Language Testing 20(1): 89–110.
Boulet J. R., van Zanten M., McKinley D. W. and Gary, N. E. (2001) Evaluating the spoken English proficiency of graduates of foreign medical schools. Medical Education 35(8): 767–773.
Brown, A. (2003) Interviewer variation and the co-construction of speaking proficiency. Language Testing 20(1): 1–25.
Bygate, M. (1999) Quality of language and purpose of task: patterns of learners’ language on two oral communication tasks. Language Teaching Research 3(3): 185–214.
Chafe, W. (1982) Integration and involvement in speaking, writing and oral literature. In D. Tannen (ed.) Spoken and Written Language: exploring orality and literacy. Norwood, NJ: Ablex.
Cunnigham-Andersson, U. (1997) Native speaker reactions to non-native speech. In A. James and J. Leather (eds) Second Language Speech: structure and process 133–44. (Studies on Language Acquisition 13). Berlin and New York: Mouton de Gruyter.
Douglas, D. (2001) Performance consistency in second language acquisition and language testing research: a conceptual gap. Second Language Research 17(4): 442–65.
Foster, P., Tonkyn, A. and Wigglesworth, G. (2000) Measuring spoken language: a unit for all reasons. Applied Linguistics 21(3): 354–75.
Fulcher, G. and Márquez Reiter, R. (2003) Task difficulty in speaking tests. Language Testing 20(3): 321–44.
Gaskell, M. G. and Marslen-Wilson, W. D. (1999) Ambiguity, competition and blending in spoken word recognition. Cognitive Science 23(4): 439–62.
Goody, J. (1977) The Domestication of the Savage Mind. Cambridge: Cambridge University Press.
Goody, J. (1987) The Interface between the Written and the Oral. Cambridge: Cambridge University Press.
Goody, J. (2000) The Power of the Written Tradition. Washington, DC: Smithsonian Institute.
Halliday, M. A. K. (1989) Spoken and Written Language. Oxford: Oxford University Press.
Hughes, R. (1996) English in Speech and Writing: investigating language and literature. London: Routledge.
Hughes, R. (2002) Teaching and Researching Speaking. London: Longman-Pearson.
Hughes, R. (forthcoming a) Assessing speaking: language proficiency versus communicative ability. Proceedings of the Second International Conference on Speech, Writing and Context. Kansai Gaidai: Kansai Gaidai Press.
Hughes, R. (ed.) (forthcoming b) Spoken Language, Applied Linguistics and TESOL: challenges for theory and practice. London: Palgrave-Macmillan.
IELTS (2001) IELTS: International English Language Testing System Handbook (July).
Johnson, M. and Tyler A. (1998) Re-analysing the OPI: how much does it look like natural conversation. In R. Young and A. Weiyn He (eds) Talking and Testing: discourse approaches to the assessment of oral proficiency 27–51. (Studies in Bilingualism 14) Amsterdam and Philadelphia: John Benjamins Publishing Company.
Lazaraton, A. (1992) The structural organisation of a language interview: a conversation analytic perspective. System 20: 373–86.
Leech, G. (2000) Grammars of spoken English, new outcomes of corpus-oriented research. Language Learning 50(4): 675–724.
Lindemann, S. (2002) Listening with an attitude: a model of native-speaker comprehension of non-native speakers in the United States. Language in Society 31(3): 419–41.
Lindemann, S. (2003) Koreans, Chinese, or Indians? Attitudes and ideologies about nonnative English speakers in the United States. Journal of Sociolinguistics 7(3): 348–64.
Lindemann, S. and Mauranen, A. (2001) ‘It’s just real messy’: the occurrence and function of just in a corpus of academic speech. English for Specific Purposes 20: 459–75.
Lu, Y. (2003) Insights into the FCE speaking test. Retrieved from www.cambridgeESOL. org/rs_notes
Luk, J. and Lin, A. (forthcoming) Uncovering the sociopolitical situatedness of accents in English as a second/foreign language (ESL/EFL) speaking assessment. In R. Hughes (ed.) Spoken Language, Applied Linguistics and TESOL: challenges for theory and practice. London: Palgrave-Macmillan.
Lumley, T. (1998) Perceptions of language-trained raters and occupational experts in a test of occupational English language proficiency. English for Specific Purposes 17(4): 347–67.
Mackenzie, C. (2000) Adult spoken discourse: the influences of age and education. International Journal of Language and Communication Disorders 35(2): 269–85.
Malvern, D. and Richards, B. (2002) Investigating accommodation in language proficiency interviews using a new measure of lexical density. Language Testing 19(1): 85–104.
Marslen-Wilson, W. (2003) Speech and language. Foresight Cognitive Systems Project. (Research review) Available at http://www.foresight.gov.uk/
Myford, C. and Wolfe, E. (2000) Monitoring sources of variability within the Test of Spoken English Assessment System. TOEFL Research Report Series. Princeton, New Jersey: Educational Testing Service.
O’Loughlin, K. (2002) The impact of gender in oral proficiency testing. Language Testing 19(2): 169–92.
Olsen, D. R. (1977) From utterance to text: the bias of language in speech and writing. Harvard Educational Review 47(3): 257–81.
Orr, M. (2002) The FCE speaking test: using rater reports to help interpret test scores. System 30(2): 143–54.
O’Sullivan, B. (2000) Exploring gender and oral proficiency interview performance. System 28(3): 373–86.
Riggenbach, H. (1998) Evaluating learner interactional skills: conversation at the micro level. In R. Young and A. Weiyun He (eds) Talking and Testing: discourse approaches to the assessment of oral proficiency 53–67. (Studies in Bilingualism 14) Amsterdam and Philadelphia: John Benjamins Publishing Company.
Rodekohr, R. K. and Haynes, W. O. (2001) Differentiating dialect from disorder: a comparison of two processing tasks and a standardized language test. Journal of Communication Disorders 34: 255–72.
Upshur, J. A. and Turner, C. E. (1999) Systematic effects in the rating of second-language speaking ability: test method and learner discourse. Language Testing 16(1): 82–116.
Verhoven, L., Aparici, M., Cahana-Amitay, D., van Hell, J. G., Kirz, S. and Viguié-Simon, A. (2002) Clause packaging in writing and speech: a cross-linguistic developmental analysis. In R. A. Berman and L. Verhoeven (eds) Cross-Linguistic Perspectives on the Development of Text-Production Abilities in Speech and Writing. Part 2. Special issue of Written Language & Literacy 5(2): 135–61.
Wennerstrom, A. (2001) Intonation and evaluation in oral narratives. Journal of Pragmatics. 33: 1183–206.
Published
2004-12-05
Issue
Section
Articles
License
Equinox Publishing Ltd.
How to Cite
Hughes, R. (2004). Testing the visible: literate biases in oral language testing. Journal of Applied Linguistics and Professional Practice, 1(3), 295-309. https://doi.org/10.1558/japl.v1.i3.295
