CALICO Journal, Vol 21, No 2 (2004)

Test Takers' Experiences with Computer- administered Listening Comprehension Tests: Interviewing for Qualitative Explorations of Test Validity

Greta Gorsuch
Issued Date: 7 Aug 2014


In this study, retrospective interviews were used to investigate reliability (and thus validity) threats to a computerized ESL listening comprehension test administered at a university in the US. The participants in the investigation, six international graduate students, were asked to respond to semi- and open-ended questions during individual interviews. Their responses were used to confirm a model of potential sources of systematic and random error variance and to generate ideas about unanticipated sources of systematic and random error variance for further investigation. The results suggested that the specific retrospective interview methodology proposed in the report was useful, dependable, and credible in generating broad based, yet focused information specific to the use of a particular test. Based on the results, further areas of research into the inescapable and complex issues of test-taker attributes as they interact with test method and conditions were suggested.

Download Media

PDF Subscribers Only

DOI: 10.1558/cj.v21i2.339-371


  • There are currently no refbacks.

Equinox Publishing Ltd - 415 The Workstation 15 Paternoster Row, Sheffield, S1 2BX United Kingdom
Telephone: +44 (0)114 221-0285 - Email:

Privacy Policy