CALICO Journal, Vol 32, No 2 (2015)

Quantitative considerations for improving replicability in CALL and applied linguistics

Luke Plonsky
Issued Date: 23 Apr 2015


There are a number of methodological practices commonly employed by CALL researchers that limit progress in the field. Some of these practices are particular to replication research, but most are more general and are found throughout the field. I describe in this paper two studies that are fabricated but that resemble much of what is found in published second language research. Each study corresponds to and contains a set of methodological issues. Following each study, I address the issues they illustrate, providing comments and suggestions for how the analyses could be modified to produce greater replicability and/or replicational value. I conclude with a summary of suggestions for quantitative reforms related to improving replication research and quantitative practices more generally in CALL and applied linguistics.

Download Media

PDF Subscribers Only

DOI: 10.1558/cj.v32i2.26857


Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.

Gass, S., & Valmori, L. (in press). Replication in interaction and working memory research: Révész (2012) and Goo (2012). Language Teaching.

Godfroid, A., & Spino, L. (in press). Reconceptualizing reactivity research: Absence of evidence is not evidence of absence. Language Learning.

Jeon, E. H., & Yamashita, J. (2014). L2 reading Comprehension and its correlates: A meta-analysis. Language Learning, 64 (1), 160–212.

Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge.

Larson-Hall, J., & Plonsky, L. (in press). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. In J. M. Norris, S. Ross, & R. Schoonen (Eds), Improving and extending quantitative reasoning in second language research. Malden, MA: Wiley.

Norris, J. M. (in press). Statistical significance testing in second language research: Basic problems and suggestions for reform. In J. M. Norris, S. Ross, & R. Schoonen (Eds.), Improving and extending quantitative reasoning in second language research. Malden, MA: Wiley.

Norris, J. M., & Ortega, L. (2006). The value and practice of research synthesis for language learning and teaching. In J. M. Norris & L. Ortega (Eds), Synthesizing research on language learning and teaching, 3–50. Philadelphia, PA: John Benjamins.

Norris, J. M., Plonsky, L., Ross, S. J., & Schoonen, R. (in press). Guidelines for reporting quantitative methods and results in primary research. Language Learning.

Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85–110.

Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics, 116–132. New York: Cambridge University Press.

Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655–687.

Plonsky, L. (in press). Statistical power, p values, descriptive statistics, and effect sizes: A ‘back-to-basics’ approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research. New York: Routledge.

Plonsky, L., & Oswald, F. L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64 (4), 878–912.

Porte, G. (2013). Who needs replication research? CALICO Journal, 30, 10–15.

Smith, B., & Schulze, M. (2013). Thirty years of the CALICO Journal – replicate, replicate, replicate. CALICO Journal, 30 (1), i–iv.


  • There are currently no refbacks.

Equinox Publishing Ltd - 415 The Workstation 15 Paternoster Row, Sheffield, S1 2BX United Kingdom
Telephone: +44 (0)114 221-0285 - Email: [email protected]

Privacy Policy