Could a multimodal dictionary serve as a learning tool? An examination of the impact of technologically enhanced visual glosses on L2 text comprehension




animated image, explicit learning, multimodal glosses, online dictionary, prepositions, second vocabulary acquisition


This study examines the efficacy of a multimodal online bilingual dictionary based on cognitive linguistics in order to explore the advantages and limitations of explicit multimodal L2 vocabulary learning. Previous studies have examined the efficacy of the verbal and visual representation of words while reading L2 texts, concluding that it facilitates incidental word retention. This study explores other potentials of multimodal L2 vocabulary learning: explicit learning with a multimodal dictionary could enhance not only word retention, but also text comprehension; the dictionary could serve not only as a reference tool, but also as a learning tool; and technology-enhanced visual glosses could facilitate deeper text comprehension. To verify these claims, this study investigates the multimodal representations’ effects on Japanese students learning L2 locative prepositions by developing two online dictionaries, one with static pictures and one with animations. The findings show the advantage of such dictionaries in explicit learning; however, no significant differences are found between the two types of visual glosses, either in the vocabulary or in the listening tests. This study confirms the effectiveness of multimodal L2 materials, but also emphasizes the need for further research into making the technologically enhanced materials more effective.


Download data is not yet available.

Author Biography

Takeshi Sato, Tokyo University of Agriculture and Technology

Takeshi Sato is an Associate Professor at the Division of Language and Culture Studies, Tokyo University of Agriculture and Technology, Japan. His academic interests include L2 vocabulary acquisition, computer & mobile-assisted language learning, and English as a Lingua Franca pedagogy.


Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning and Technology, 5(1), 202-232.

Boers, F. (2004). Expanding learners' vocabulary through metaphor awareness: What expansion, what learners, what vocabulary? In M. Achard and S. Niemeier (Eds.), Cognitive Linguistics, Second Language Acquisition, and Foreign Language Teaching (pp. 211-232). Berlin: Mouton de Gruyter.

Boers, F., & Lindstromberg, S. (2008). How cognitive linguistics can foster effective vocabulary teaching. In F. Boers, & S. Lindstromberg (Eds.), Cognitive linguistic approaches to teaching vocabulary and phraseology (pp.1-64). Berlin: Mouton de Gruyter.

Boers, F. (2013). Cognitive linguistic approaches to teaching vocabulary: Assessment and integration. Language Teaching, 46, 208-24.

Chapelle, C. A. (1998). Multimedia CALL: Lessons to be learned from research on instructed SLA. Language Learning & Technology, 2(1), 22-34.

Cho, K. (2010). Fostering the acquisition of English prepositions by Japanese learners with networks and prototypes. In S. de Knop, F. Boers, & T. De Rycker (Eds.), Fostering language teaching efficiency through cognitive linguistics (pp. 259–276). Berlin, New York: Mouton de Gruyter.

Chun, D. L., & Plass, J. L. (1996). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80(2), 183-198 .

Dewell, R. B. (1994). Over again: Image-schema transformations in semantic analysis. Cognitive Linguistics, 5, 351-380.

Ellis, N. C. (1995). The psychology of foreign language vocabulary acquisition: Implication for CALL. Computer Assisted Language Learning 8(2-3), 103-128.

Groot, P. (2000). Computer assisted second language vocabulary acquisition. Language Learning and Technology 4(1), 60-81.

Johnson, M. (1987). Body in the mind. Chicago: University of Chicago Press.

Keane, J.A., Nkwenti-Azeh, B., and Singleton, J.A. (1997). Design considerations for a CALL conceptual dictionary. Computer Assisted Language Learning. 10(4), 363-374.

Lakoff, G. (1987). Woman, fire and dangerous thing. Chicago: University of Chicago Press.

Langacker, R, W. (1987). Foundations of cognitive grammar, Volume I, theoretical prerequisites. Stanford, California: Stanford University Press.

Lin, C. & Tseng, Y. (2012). Videos and animations for vocabulary learning: A study on difficult words. TOJET: The Turkish Online Journal of Educational Technology, 11(4), 346-355.

Lindstromberg, S. (1996). English prepositions explained. Amsterdam: Benjamins.

Lindstromberg, S. (2001). Preposition entries in UK monolingual learner's dictionaries: Problems and possible solutions. Applied Linguistics, 2 (1), 79-103.

Littlemore, J. (2009). Applying cognitive linguistics to second language learning and teaching. Basingstoke: Palgrave Macmillan.

Lomicka, L. L. (1998). "To gloss or not to gloss": An investigation of reading comprehension online. Language Learning & Technology, 1, 41-50.

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389-401.

Ngu, B, H. & Rethinasamy, S. (2006), Evaluating a CALL software on the learning of English prepositions. Computer & Education, 47, 41-55.

Morimoto, S., & Loewen, S. (2007). A comparison of the effects of image-schema-based instruction and translation-based instruction on the acquisition of L2 polysemous words. Language Teaching Research, 11(3), 347–372.

Pachler, N. (2001). Electronic reference tools for foreign language learners, teachers and users: Offline vocabulary look-up programs. Language Learning Journal, 24, 24-29.

Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart, and Winston.

Sato, T., & Suzuki, A. (2010). Do multimedia-oriented visual glosses really facilitate EFL vocabulary learning?: A comparison of planar images with three- dimensional images. Asian EFL Journal, 12(4), 160-172.

Sato, T., & Suzuki A. (2011). Verifying Multimedia Gloss: Image schema and polysemous vocabulary in English. Proceedings of the EuroCALL 2011 Conference, 285-293.

Sato, T., Lai Y., & Burden, T. (2014). Examining the impact of individual differences of information processing styles in technology-enhanced second vocabulary learning. Proceedings of CLaSIC 2014, 432-440.

Sundberg, P.A. (1998) Animation in CALL: Learning to think in the fourth dimension. In CALICO'98 Symposium (7/1998), San Diego, California. Retrieved 24th September 2016 from

Tanaka, S. (1990). Ninchi imiron [Cognitive Semantics]. Tokyo: San'yusha.

Tanaka, S., Takeda, S., & Kawade, S. (Eds.) (2003). E-gate English-Japanese dictionary. Tokyo: Benesse Corporation.

Yasuda, S. (2010). Learning phrasal verbs through conceptual metaphors: A case of Japanese EFL learners. TESOL Quarterly, 44(2), 250–273.

Yeh, Y., & Wang, C. (2003). Effects of multimedia vocabulary annotations and learning styles on vocabulary learning. CALICO Journal, 21(1), 131-144.

Yoshii, M., & Fraitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture annotation types. CALICO Journal, 20(1), 33-58.

Zwaan, R, A, & Radvansky, G, A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123(2), 162-185.






Research papers