Using Formative Evaluation in the Development of Learner-centered Materials

Authors

  • Carl S. Blyth
  • James N. Davis

DOI:

https://doi.org/10.1558/cj.v25i1.48-68

Keywords:

Instructional Technology Development, Formative Evaluation, Contextualized Versus Decontextualized Language, Student Data, Learner-centered Curriculum

Abstract

In this article we report on an 8-year process that included three successive iterations of the following cycle: (a) development of instructional technology, (b) formative evaluation, and (c) modification of the technology. From the first formative evaluation to the last, our students told us that they found heavily contextualized language difficult to learn and frequently requested more decontextualized language for textbook presentations and for practice. With the aid of formative evaluation data (e.g., performance data based on think aloud protocols, attitudinal data, retrospective interviews, and course surveys), we tried to strike a balance between what students said they wanted (i.e., more decontextualized language input and practice) and what we as language teachers and curriculum developers believed that they needed (i.e., more contextualized language input and practice). Three theoretical constructs proved particularly relevant in helping us interpret the data: activity theory (Lantolf & Appel, 1994; Lantolf, 2000); the naïve lexical hypothesis (Bland, Noblitt, Armington, & Gay, 1990); and the lexical approach (Lewis, 1993), also known as the lexical syllabus (Willis, 1990). We argue that when formative evaluation becomes a central part of the development of instructional technology, the results are a more learner-centered curriculum with more user-friendly technology.

References

Adair-Hauck, B., Willingham-McLain, L., & Youngs, B. E. (2000). Evaluating the integration of technology and second language learning. CALICO Journal, 17 (2), 269-306.

Allen, W., & Fouletier-Smith, N. (1995). Parallèles: Communication et culture. Englewood Cliffs, NJ: Prentice-Hall.

Bernstein, J., Najmi, A., & Ehsani, F. (1999). Subarashii: Encounters in Japanese spoken language education. CALICO Journal, 16 (3), 361-384.

Bland, S., Noblitt, J., Armington, S., & Gay, G. (1990). The Naïve Lexical Hypothesis: Evidence from computer-assisted language learning. Modern Language Journal, 74 (4), 440-450.

Blyth, C. (1999). Implementing technology in the foreign language curriculum: Redefining the boundaries between language and culture. Journal of Educational Computing Research, 20 (1), 39-58.

Chenoweth, N. A., & Murday, K. (2003). Measuring student learning in an online French course. CALICO Journal, 20 (2), 285-314.

Coniam, D. (2002). Perceptions of a multimedia syllabus—Making the demands of a performance test more accessible. System, 31 (1), 55-70.

Crooks, B., & Lamy, M.-N. (1995). Using combinations of video, audio and print to teach French at a distance: A case study of the formative evaluation cycle. European Journal of Psychology of Education, 10 (2), 131-144.

Dansuwan, S., Nishina, K., Akahori, K., & Shimizu, Y. (2001). Development and evaluation of a Thai learning system on the web using natural language processing. CALICO Journal, 19 (1), 67-88.

Ehrmann, S. (1995). Asking the hard questions about technology use and education. Change, 27 (2), 20-27.

Ericsson, K. A., & Simon, H. A. (1994). Protocol analysis: Verbal reports as data. Cambridge: MIT Press.

George, J., & Cowan, J. (1999). A handbook of techniques for formative evaluation: Mapping the student’s learning experience. London: Kogan Page.

Holland, V. M., Kaplan, J. D., & Sabol, M. A. (1999). Preliminary tests of language learning in a speechinteractive graphics microworld. CALICO Journal, 16 (3), 339-359.

Kabata, K., & Yang, X. J. (2002). Developing multimedia lesson modules for intermediate Japanese. CALICO Journal, 19 (3), 563-570

Lantolf, J. (Ed.). (2000). Sociocultural theory and second language learning. New York: Oxford University Press.

Lantolf, J., & Appel, G. (Eds.). (1994). Vygotskyan approaches to second language research. Norwood NJ: Ablex Publishing Company.

Lewis, M. (1993). The lexical approach. Hove, England: Language Teaching Publication.

McCormick, C. P. (1997). A case study of student protocols in the development of foreign language learning software. Unpublished dissertation, University of Texas at Austin.

Moudraia, O. (2001). Lexical approach to second language teaching. (ERIC Document Reproduction Service No. ED455698.) Retrieved March 27, 2007, from http://www.cal.org/resources/digest/0102lexical.html

Noblitt, J., & Bland, S. (1991). Tracking the learner in computer-aided language learning. In B. Freed (Ed.), Foreign language acquisition research and the classroom (pp. 120-131). Lexington, MA: DC Heath.

Noblitt, J. (1995). The electronic language learning environment. In C. Kramsch (Ed.), Redefining the boundaries of language study (AAUSC issues in language program direction) (pp. 263-292). Boston: Heinle.

Patton, M. Q. (2001). Evaluation, knowledge management, best practices and higher quality lessons learned. American Journal of Evaluation, 22 (3), 329-336.

Rypa, M. E., & Price, P. (1999). VILTS: A tale of two technologies. CALICO Journal, 16 (3), 385-404.

Sarkissian, J. (2001). Monitoring student behavior in computerized Latin exercises. CALICO Journal, 18 (2), 339-355.

Wertsch, J. V. (1985). Vygotsky and the social formation of mind. Cambridge, MA: Harvard University Press.

Wertsch, J. V. (1991). Voices of the mind: A sociocultural approach to mediated action. Cambridge, MA: Harvard University Press.

Willis, D. (1990). The lexical syllabus: A new approach to language teaching. London: Collins COBUILD. Retrieved March 27, 2007, from http://www.cels.bham.ac.uk/resources/LexSyll.htm

Zhao, Y. (2003). A comprehensive review of research on technology uses in language education. CALICO Journal, 21 (1), 7-27.

Downloads

Published

2013-01-14

Issue

Section

Articles

How to Cite

Blyth, C. S., & Davis, J. N. (2013). Using Formative Evaluation in the Development of Learner-centered Materials. CALICO Journal, 25(1), 48-68. https://doi.org/10.1558/cj.v25i1.48-68

Most read articles by the same author(s)