Testing audiovisual comprehension tasks with questions embedded in videos as subtitles: a pilot multimethod study

Juan Carlos Casañ Núñez

doi:10.4995/eurocall.2017.7062

Authors

Juan Carlos Casañ Núñez Universitat Politècnica de València https://orcid.org/0000-0001-6918-9604

DOI:

https://doi.org/10.4995/eurocall.2017.7062

Keywords:

audiovisual comprehension, listening comprehension, multimethod design, Spanish as a foreign language, subtitles, video listening test

Abstract

Listening, watching, reading and writing simultaneously in a foreign language is very complex. This paper is part of wider research which explores the use of audiovisual comprehension questions imprinted in the video image in the form of subtitles and synchronized with the relevant fragments for the purpose of language learning and testing. Compared to viewings where the comprehension activity is available only on paper, this innovative methodology may provide some benefits. Among them, it could reduce the conflict in visual attention between watching the video and completing the task, by spatially and temporally approximating the questions and the relevant fragments. The technique is seen as especially beneficial for students with a low proficiency language level.

The main objectives of this study were to investigate if embedded questions had an impact on SFL students’ audiovisual comprehension test performance and to find out what examinees thought about them. A multimethod design (Morse, 2003) involving the sequential collection of three quantitative datasets was employed. A total of 41 learners of Spanish as a foreign language (SFL) participated in the study (22 in the control group and 19 in the experimental one). Informants were selected by non-probabilistic sampling. The results showed that imprinted questions did not have any effect on test performance. Test-takers’ attitudes towards this methodology were positive. Globally, students in the experimental group agreed that the embedded questions helped them to complete the tasks. Furthermore, most of them were in favour of having the questions imprinted in the video in the audiovisual comprehension test of the final exam. These opinions are in line with those obtained in previous studies that looked into experts’, SFL students’ and SFL teachers’ views about this methodology (Casañ Núñez, 2015a, 2016a, in press-b). On the whole, these studies suggest that this technique has potential benefits for FL learning and testing. Finally, the limitations of the study are discussed and some directions for future research are proposed.

Downloads

Download data is not yet available.

References

Alderson, J. C. (2005). Diagnosing Foreign Language Proficiency: The Interface Between Learning and Assessment. New York: Continuum.

Alderson, J. C., Clapham, C. & Wall, D. (1995). Language Test Construction and Evaluation. Cambridge: Cambridge University Press.

Allen, I. E., & Seaman, C. A. (2007). Likert Scales and Data Analyses. Quality Progress, 40(7), 64-65.

Batty, A. O. (2014). A Comparison of Video- and Audio-mediated Listening Tests with Many-Facet Rasch Modeling and Differential Distractor Functioning. Language Testing, 32(1), 3–20. doi:

1177/0265532214531254.

Bejar, I., Douglas, D., Jamieson, J., Nissan, S. & Turner, J. (2000). TOEFL 2000 Listening Framework: A Working Paper. Princeton, New Jersey: Educational Testing Service. Retrieved from

http://ets.org/Media/Research/pdf/RM-00-07.pdf.

Bisquerra, R. & Pérez-Escoda, N. (2015). ¿Pueden las escalas Likert aumentar en sensibilidad? REIRE, Revista d’Innovació i Recerca en Educació, 8(2), 129-147. doi:10.1344/reire2015.8.2828.

Briggs, S. R. & Cheek, J. M. (1986). The Role of Factor Analysis in the Development and Evaluation of Personality Scales. Journal of Personality, 54(1), 106-148.

Brown, J. D. (2001). Using Surveys in Language Programs. Cambridge: Cambridge University Press.

Buck, G. (2001): Assessing Listening. Cambridge: Cambridge University Press.

Casañ Núñez, J. C. (2015a). Un marco teórico sobre el uso de preguntas de comprensión audiovisual integradas en el vídeo como subtítulos: un estudio mixto. MarcoELE, 20, 1-45. Retrieved from http://marcoele.com/comprension-audiovisual-y- subtitulos/.

Casañ Núñez, J. C. (2015b). Subtitulación de preguntas de comprensión audiovisual: ejemplificación en una secuencia de Ópera prima de Fernando Trueba. Foro de profesores de ELE, 11, 45-56. Retrieved from https://ojs.uv.es/index.php/foroele/article/view/7095.

Casañ Núñez, J. C. (2016a). Actividades de comprensión audiovisual con preguntas integradas en forma de subtítulos: la opinión de catorce estudiantes universitarios de español lengua extranjera. Skopos, 7, 19-38.

Casañ Núñez, J. C. (2016b). Desarrollo de una prueba de comprensión audiovisual. MarcoELE, 22, 1-70. Retrieved from http://marcoele.com/descargas/22/casan- prueba_audiovisual.pdf.

Casañ Núñez, J. C. (in press-a). Diseño y fiabilidad de un cuestionario sobre la comprensión auditiva/audiovisual. Bellaterra Journal of Teaching & Learning Language & Literature.

Casañ Núñez, J. C. (in press-b). Tareas de comprensión audiovisual con preguntas subtituladas: valoraciones de cinco profesores universitarios de español como lengua extranjera. E-JournALL,

EuroAmerican Journal of Applied Linguistics and Languages.

Celce-Murcia, M. & Olshtain, E. (2000). Discourse and Context in Language Teaching: A Guide for Language Teachers. Cambridge: Cambridge University Press.

Cohen, L., Manion, L. & Morrison, K. (2011). Research Methods in Education (7th ed.). New York: Routledge.

Coniam, D. (2001). The Use of Audio or Video Comprehension as an Assessment Instrument in the Certification of English Language Teachers: A Case Study. System, 29(1), 1-14. doi:

1016/S0346-251X(00)00057-9.

Council of Europe (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press.

De Vaus, D. A. (2002). Surveys in Social Research (5th ed.). New South Wales: Allen & Unwin.

Dörnyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University Press. Field, A. (2009). Discovering Statistics Using SPSS (3rd ed.). London: Sage Publications. Field, J.

(2008). Listening in the Language Classroom. Cambridge: Cambridge University Press.

Flowerdew, J. & Miller, L. (2005). Second Language Listening. New York: Cambridge University Press.

González-Vera, P. & Hornero Corisco, A. (2016). Audiovisual Materials: A Way to Reinforce Listening Skills in Primary School Teacher Education. Language Value, 8(1), 1-

doi: 10.6035/LanguageV.2016.8.2. Retrieved from http://www.e-revistes.uji.es/languagevalue.

Green, R. (2013). Statistical Analyses for Language Testers. New York: Palgrave.

Gruba, P. (1993). A Comparison Study of Audio and Video in Language Testing. JALT Journal, 15(1), 85-88.

Gruba, P. (1997). The Role of Video Media in Listening Assessment. System, 25(3), 335-345. doi:10.1016/s0346-251x(97)00026-2.

Harris, T. (2003). Listening with Your Eyes: The Importance of Speech-Related Gestures in the Language Classroom. Foreign Language Annals, 36(2), 180-187. doi: 10.1111/j.1944-9720.2003.tb01468.x.

Hidalgo de la Torre, R. (Coord.), (2013). Prepara y practica el DELE B1. Barcelona: Octaedro.

Jacoby, J. & Matell, M. S. (1971). Three-Point Likert Scales Are Good Enough. Journal of Marketing Research, 8, 495-500.

Krashen, S. D. & Terrel, T. D. (1983). The Natural Approach: Language Acquisition in the Classroom. Oxford: Pergamon Press.

Larson-Hall, J. (2010). A Guide to Doing Statistics in Second Language Research Using SPSS. New York: Routledge. doi: 10.4324/9780203875964.

Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology, 22(140), 1-55.

Londe, Z. C. (2009). The Effects of Video Media in English as a Second Language Listening Comprehension Tests. Issues in Applied Linguistics, 17(1), 41-50.

Lynch, T. (2009). Teaching Second Language Listening. Oxford: Oxford University Press.

Lynch, T. (2012). Traditional and Modern Skills. Introduction. In M. Eisenmann & T. Summer (Eds.), Basic Issues in EFL Teaching and Learning (pp. 69-81). Heidelberg: Winter.

Mayer, R. (2001). Multimedia Learning. Cambridge: Cambridge University Press.

Mayer, R. (2014). The Cambridge Handbook of Multimedia Learning. (2nd ed.). Cambridge: Cambridge University Press.

Martín Peris, E. (2007). La didáctica de la comprensión auditiva. MarcoELE, 5. Retrieved from http://marcoele.com/la-didactica-de-la-comprension-auditiva/ (Original work published in 1991).

Mendelshon, D. J. (1994). Learning to Listen: a Strategy Based Approach for the Second Language Learner. Carlsbad, California: Dominie Press.

Morse, J. M. (2003). Principles of Mixed Methods and Multimethod Research Design. In A. Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in Social and Behavioral Research (pp.

-208). Thousand Oaks, California: Sage.

Ockey, G. J. (2007). Construct Implications of Including Still Image or Video in Computer-Based Listening Tests. Language Testing, 24(4), 517-537. doi: 10.1177/0265532207080771.

Osborne, J. W. & Overbay, A. (2004). The Power of Outliers (and Why Researchers Should Always Check for Them). Practical Assessment, Research & Evaluation, 9(6), 1-12. Retrieved from http://PAREonline.net/getvn.asp?v=9&n=6.

Pallant, J. (2011). SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS (4th ed.). Maidenhead: Open University Press/McGraw-Hill.

Pardo-Ballester, C. (2016). Using Video in Web-Based Listening Tests. Journal of New Approaches in Educational Research, 5(2), 91-98. doi: 10.7821/naer.2016.7.170.

Plass, J. L. & Jones, L. (2005). Multimedia Learning in Second Language Acquisition. In R. E. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (pp. 467-488). Cambridge:

Cambridge University Press.

Riley, P. (1979). Viewing Comprehension: “L’Oeil Écoute”. Mélanges CRAPEL, 10, 80-95. Retrieved from http://www.atilf.fr/IMG/pdf/melanges/6riley.pdf.

Rubin, J. (1995a). The Contribution of Video to the Development of Competence in Listening. In D. J. Mendelsohn & J. Rubin (Eds.), A Guide for the Teaching of Second Language Listening (pp.

-165). San Diego, California: Dominie Press, Inc.

Rubin, J. (1995b). An Overview to A Guide for the Teaching of Second Language Listening. In D. J. Mendelsohn & J. Rubin (Eds.), A Guide for the Teaching of Second Language Listening (pp. 7-11). San Diego, California: Dominie Press, Inc.

Saris, W. E. & Gallhofer, I. N. (Eds.). (2014). Design, Evaluation, and Analysis of Questionnaires for Survey Research (2nd ed.). doi:10.1002/9781118634646.

Stempleski, S. & Tomalin, B. (2001). Film. Oxford: Oxford University Press.

Stoller, F. L. (1992). Using Video in Theme-Based Curricula. In Susan Stempleski & Pual Arcario (Eds.), Video in Second Language Teaching: Using, Selecting and Producing Video for the Classroom (pp. 25-46). New York: TESOL.

Sueyoshi, A. & Hardison, D. M. (2005). The Role of Gestures and Facial Cues in Second Language Listening Comprehension. Language Learning, 55(4), 661-669. doi: 10.1111/j.0023-8333.2005.00320.x.

Suhr, D. & Shay, M. (2009). Guidelines for Reliability, Confirmatory and Exploratory Factor Analysis. In Conference Proceedings of the Western Users of SAS Software (pp. 1-15). San Jose,

California. Retrieved from http://www.lexjansen.com/wuss/2009/anl/ANL-SuhrShay.pdf.

Suvorov, R. S. (2008). Context Visuals in L2 Listening Tests: the Effectiveness of Photographs and Video vs. Audio-Only Format. Retrospective Theses and Dissertations (paper 15448). Retrieved from http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=16447&context=rtd.

Suvorov, R. S. (2015). The Use of Eye Tracking in Research on Video-Based Second Language (L2) Listening Assessment: A Comparison of Context Videos and Content Videos. Language Testing, 32(4), 463-483. doi: 10.1177/0265532214562099.

Sweller, J., Ayres, P. & Kalyuga, S. (2011). Cognitive Load Theory. London: Springer. Underwood, M. (1989). Teaching Listening. London: Longman. Ur, P. (1994). Teaching Listening Comprehension (12th printing): Cambridge: Cambridge University Press.

Ur, P. (1999). A Course in Language Teaching: Practice and Theory. Cambridge: Cambridge University Press.

Vandergrift, L. & Goh, C. C. M. (2012). Teaching and Learning Second Language Listening. New York: Routledge.

Wagner, E. (2007). Are They Watching? Test-Taker Viewing Behavior During an L2 Video Listening Test. Language Learning and Technology, 11(1), 67-86. Retrieved from http://llt.msu.edu/vol11num1/pdf/wagner.pdf.

Wagner, E. (2008). Video Listening Tests: What Are They Measuring? Language Assessment Quarterly, 5(3), 218-243. doi: 10.1080/15434300802213015.

Wagner, E. (2010a). The Effect of the Use of Video Texts on ESL Listening Test-Taker Performance. Language Testing, 27(4), 493–513. doi: 10.1177/0265532209355668.

Wagner, E. (2010b). Test-Takers’ Interaction with an L2 Video Listening Test. System, 38(2), 280-291. doi: 10.1016/j.system.2010.01.003.

Wagner, E. (2013). An Investigation of How the Channel of Input and Access to Test Questions Affect L2 Listening Test Performance. Language Assessment Quarterly, 10(2), 178–195. doi:

1080/15434303.2013.769552.