Constructing an evidence-base for future CALL design with ‘engineering power’: The need for more basic research and instrumental replication

Zöe Handley

Abstract

This paper argues that the goal of Computer-Assisted Language Learning (CALL) research should be to construct a reliable evidence-base with ‘engineering power’ and generality upon which the design of future CALL software and activities can be based. In order to establish such an evidence base for future CALL design, it suggests that CALL research needs to move away from CALL versus non-CALL comparisons, and focus on investigating the differential impact of individual attributes and affordances, that is, specific features of a technology which might have an impact on learning. Further, in order to help researchers find possible explanations for the success or failure of CALL interventions and make appropriate adjustments to their design, it argues that these studies should be conducted within the framework of Second Language Acquisition (SLA) theory and research. Despite this, a recent review of research examining the effectiveness of CALL in primary and secondary English as a Foreign Language (EFL) found that CALL vs. non-CALL comparisons are still common and studies focusing on individual coding elements are rare. Further, few studies make links with SLA and few measure linguistic outcomes using measures developed in the field of SLA.  One reason for this may be poor reporting of methods and difficulty in obtaining the instruments used in SLA research. Reporting guidelines and the use of the IRIS database (www.iris-databse.org) are introduced as possible solutions to these problems.

Keywords

Research methods; basic research; second language acquisition; replication; instruments

Full Text:

HTML PDF

References

Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32(1), 21-24. http://dx.doi.org/10.3102/0013189X032001021

Barab, S. & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1-14. http://dx.doi.org/10.1207/s15327809jls1301_1

Borg, S. (2003). Teacher cognition in language teaching: A review of research on what language teachers think, know, believe and do. Language Teaching, 36(2), 81-109. http://dx.doi.org/10.1017/S0261444803001903

Burkhardt, H. & Schoenfeld, A. H. (2003). Improving educational research: Toward a more useful, more influential, and better-funded enterprise. Educational Researcher, 32(3), 3-14. http://dx.doi.org/10.3102/0013189X032009003

Burston, J (2006). Working towards effective assessment of CALL. In Donaldson, P. R. & Haggstrom, M. A. (eds.). Changing Language Education Through CALL. London: Routledge, 249-270.

Campbell, M., Fitzpatrick, R., Haines, A., Kinmouth, A. L., Sandercock, P., Spiegelhalter, D. & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. British Medical Journal, 321, 694-696. http://dx.doi.org/10.1136/bmj.321.7262.694

Chapelle, C. (2010). The spread of computer-assisted language learning. Language Teaching, 43(1), 66-74.http://dx.doi.org/10.1017/S0261444809005850

Chen, C.-M. & Li, Y. L. (2010). Personalised context-aware ubiquitous learning system for supporting effective English as a second language. TESOL Quarterly, 20(1), 27-46.

Chun, D. (2012). Review article: Replication studies in CALL research. CALICO Journal, 29(4), 591-600. http://dx.doi.org/10.11139/cj.29.4.591-600

Colpaert, J. (2006). Pedagogy-driven design for online language teaching and learning. CALICO Journal, 23(3), 477-497.

Colpaert, J. (2010). Elicitation of language learners' personal goals as design concepts. Innovation in language learning and teaching, 4(3) 259-274. http://dx.doi.org/10.1080/17501229.2010.513447

Dalton, B., Proctor, C. P., Uccelli, P., Mo, E. & Snow, C. E. (2011). Designing for diversity: The role of reading strategies and interactive vocabulary in a digital reading environment for fifth-grade monolingual English and bilingual students. Journal of Literacy Research, 43, 68-100 http://dx.doi.org/10.1177/1086296X10397872

Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8 http://dx.doi.org/10.3102/0013189X032001005

García Mayo, M. (2005). Interactional strategies for interlanguage communication: Do they provide evidence for attention to form? In A. Housen & M. Pierrard (Eds.), Investigations in instructed second language acquisition (Studies on Language Acquisition Series). Mouton de Gruyter. http://dx.doi.org/10.1515/9783110197372.3.383

Garrett, N. (1991). Technology in the service of language learning: Trends and issues. Modern Language Journal, 75, 74-101 http://dx.doi.org/10.1111/j.1540-4781.1991.tb01085.x

Gorard, S., Roberts, K. & Taylor, C. (2004). What kind of creature is a design experiment? British Educational Research Journal, 30(4), 577-590. http://dx.doi.org/10.1080/0141192042000237248

Gruber-Miller, J. & Benton, C. (2001). How do you say 'MOO' in Latin? Assessing student learning and motivation in beginning Latin. CALICO Journal, 18(2), 305-38.

Gutiérrez-Colon, M.; Gibert, M.I.; Triana, I.; Gimeno, A.; Appel, C. & Hopkins, J. (2003). Improving learners' reading skills through instant short messages: a sample study using WhatsApp. WorldCALL 2013 – CALL: Sustainability and Computer-Assisted Language Learning Conference Proceedings. University of Ulster, pp.80-84.

Hiebert, J., Gallimore, R. & Stigler, J. (2002). A knowledge base for the teaching profession: What would it look like and how can we get one? Educational Researcher, 31(3), 3-15. http://dx.doi.org/10.3102/0013189X031005003

Higher Education Research Funding Council (1999). Guidance on submissions research assessment exercise, Paragraph 1.12. London: Higher Education Funding Council for England and Wales 1999.

Hwang, W. Y. & Chen, H. S. L. (2013). Users' familiar situational context facilitate the practice of EFL in elementary schools with mobile devices. Computer-Assisted Language Learning, 26(2), 101-125 http://dx.doi.org/10.1080/09588221.2011.639783

Kim, D.; Gilman, D. A. (2008). Effects of Text, Audio, and Graphic Aids in Multimedia Instruction for Vocabulary Learning. Educational Technology & Society. 11 (3): 114-126.

Kumaravadivelu, B. (1994). The post-method condition: (E)merging strategies for Second/Foreign language teaching. TESOL Quarterly, 28(1), 27-48. http://dx.doi.org/10.2307/3587197

Levy, M. & Stockwell, G. (2006). CALL Dimensions: Options and Issues in Computer-Assisted Language Learning. London: Lawrence Erlbaum.

Lu, M. (2008). Effectiveness of vocabulary learning via mobile phones. Journal of Computer Assisted Learning, 24(6), 515-525. http://dx.doi.org/10.1111/j.1365-2729.2008.00289.x

Luckin, R. & Clark, W. (2011). More than a game: The participatory Design of contextualised technology-rich learning experiences with the ecology of resources. Journal of e-Learning Knowledge Society, 7(3), 33-50.

Lund, A. (2008). Wikis: A collective approach to language production. ReCALL, 20(1), 35-54. http://dx.doi.org/10.1017/S0958344008000414

Macaro, E., Handley, Z. L., & Walter, C. (2012). A systematic review of CALL in English as a second language: Focus on primary and secondary education. Language Teaching, 45(1), 1-43 http://dx.doi.org/10.1017/S0261444811000395

Marsden, E. & King, J. (2013). The Instruments for Research into Second Languages (IRIS) digital repository. The Language Teacher, 37(2), 35-38.

Marsden, E. J., & Mackey, A. (2014). IRIS: a new resource for second language research. Linguistic Approaches to Bilingualism, 4(1), 125-130. http://dx.doi.org/10.1075/lab.4.1.05mar

Mifka Profozic, N. (2012). Oral corrective feedback, individual differences and L2 acquisition of French past tenses. (Unpublished doctoral dissertation). University of Auckland.

Moher, D., Jones, A. & Lepage, L. (2001). Use of the CONSORT Statement and quality of reports of randomized trials. A comparative before-and-after evaluation. The Journal of the American Medical Association, 285, 1992-5. http://dx.doi.org/10.1001/jama.285.15.1992

Moher, D., Schulz, K. ., Altman, D. (2001). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. The Journal of the American Medical Association, 285, 1987-91. http://dx.doi.org/10.1001/jama.285.15.1987

O'Hara, & Pritchard, (2008). Hypermedia authoring as a vehicle for vocabulary development in middle school English as a second language classrooms. Clearing House: A Journal of Educational Strategies, Issues, and Ideas, 82(2), 60-65. http://dx.doi.org/10.3200/TCHS.82.2.60-65

Oswald, F. L. & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85-110. http://dx.doi.org/10.1017/S0267190510000115

Pardo-Ballester, C. & Rodriguez, J. C. (2009). Using design-based research to guide the development of online instructional materials. In Chapelle, C. A., Jun, H. G., & Katz, I. (Eds.). Developing and evaluating language learning materials (pp. 86-102). Ames, IA: Iowa State University.

Pardo-Ballester, C. & Rodriguez, J. C. (2010). Developing Spanish online readings using design-based research. CALICO Journal, 27(3), 540-553. http://dx.doi.org/10.11139/cj.27.3.540-553

Pederson, K (1987). Research on CALL. In Smith, W. F. (ed.). Modern media in foreign language education: Theory and implementation. Lincolnwood, Illinois: National Textbook Company, pp. 99-131.

Polio, C. (2012). Replication in published applied linguistics research: A historical perspective. In Porte, G. (ed.). Replication research in applied linguistics. Cambridge: Cambridge University Press, pp. 47-91.

Polio, C. & Gass, S. (1997). Replication and reporting. Studies in Second Language Acquisition, 19, 499-508. http://dx.doi.org/10.1017/S027226319700404X

Porte, G. (2013). Who needs replication? CALICO Journal, 30(1), 10-15 http://dx.doi.org/10.11139/cj.30.1.10-15

Porte, G. & Richards, K. (2012). Focus article: Replication in second language writing research. Journal of Second Language Writing, 21(2012), 284-193. http://dx.doi.org/10.1016/j.jslw.2012.05.002

Proctor, C. P., Dalton, B. & Grisham, D. L. (2007). Scaffolding English language learners and struggling readers in a universal literacy environment with embedded strategy instruction and vocabulary support. Journal of Literacy Research, 39(1), 71-93.

Révész, A. (2011). Task complexity, focus on L2 constructions, and individual differences: A classroom-based study. The Modern Language Journal, 95(4). http://dx.doi.org/10.1111/j.1540-4781.2011.01241.x

Sasaki, A. & Takeuchi, O. (2010). EFL students' vocabulary learning in NS-NNS e-mail interactions: Do they learn new words by imitation? ReCALL, 22(1): 70-82. http://dx.doi.org/10.1017/S0958344009990206

Slavin, R. E. (1995). Best evidence synthesis: An intelligent alternative to meta-analysis. Journal of Clinical Epidemiology, 48(1), 9-18. http://dx.doi.org/10.1016/0895-4356(94)00097-A

Stockwell, G. (2012). Diversity in research and practice. In Stockwell, G. (ed.). Computer-Assisted Language Learning: Diversity in Research and Practice. Cambridge: Cambridge University Press, pp. 147-163. http://dx.doi.org/10.1017/CBO9781139060981.009

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus-groups. International Journal for Quality in Health Care, 19(6), 349-357. http://dx.doi.org/10.1093/intqhc/mzm042

van Lier, L. (2004). The Ecology and Semiotics of Language Learning. A Sociocultural Perspective. Boston: Kluwer Academic http://dx.doi.org/10.1007/1-4020-7912-5

Yildiz, R. & Atkins, M. (1993). Evaluating multimedia applications. Computers in Education, 21(1/2), 133-139. http://dx.doi.org/10.1016/0360-1315(93)90056-O

Yutdhana, S. (2008). Design-based research in CALL. In Egbert, J. L. & Petrie, G. M. (eds.). CALL research perspectives. Mahwah, NJ: Lawrence Erlbaum, pp. 169-178.

Abstract Views

2209
Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


 

Cited-By (articles included in Crossref)

This journal is a Crossref Cited-by Linking member. This list shows the references that citing the article automatically, if there are. For more information about the system please visit Crossref site

1. Replication research in computer-assisted language learning: Replication of Neri et al. (2008) and Satar & Ӧzdener (2008)
Zöe Handley
Language Teaching  vol: 51  issue: 3  first page: 417  year: 2018  
doi: 10.1017/S0261444817000040



Licencia Creative Commons

This journal is licensed under a  Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Universitat Politècnica de València

e-ISSN: 1695-2618    http://dx.doi.org/10.4995/eurocall