Comparing ChatGPT and Authentic Discourse as L2 Academic Speaking Partners: A Corpus-based Analysis and Exploration of Pedagogical Applicability
Submitted: 05/07/2025
|Accepted: 10/12/2025
|Published: 12/26/2025
Copyright (c) 2025 Koki Sekitani, Masaaki Ogura, Takeshi Sato
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Downloads
Keywords:
ChatGPT, academic spoken English, corpus linguistic analysis, L2 speaking practice, syntactic complexity, readability
Supporting agencies:
Abstract:
This study examines the extent to which spoken academic English produced by the ChatGPT-4o model (ChatGPT) aligns with authentic discourse and is applicable for use in L2 speaking materials. As generative AI tools gain popularity among L2 learners for speaking practice, it is essential to assess the linguistic and discourse-level validity of such output. Ten dialogues generated by ChatGPT were compared with an authentic transcript of a doctoral dissertation defence from the Michigan Corpus of Academic Spoken English (MICASE) using a corpus linguistic and content analysis approach. Quantitative analyses revealed that ChatGPT produced language with higher lexical diversity, greater use of advanced vocabulary, and more complex syntax than the MICASE sample, but with lower readability. Content analysis showed that while ChatGPT simulated turn-taking and appropriate question–answer sequences, it lacked features of spontaneous interaction such as clarification requests, topic shifts, and overlapping speech. Speaker roles in the generated texts were consistent but followed more scripted and idealized patterns. These findings suggest that ChatGPT can serve as a useful supplementary tool for academic speaking practice, especially in providing structured, high-level input. However, its limitations in simulating authentic interaction highlight the need for guided integration alongside real human dialogue. The study is expected to contribute to the growing body of research on AI-assisted language learning.
References:
Al-Khasawneh, F. M. (2023). From text to tech: Investigating chatbots’ role in enhancing oral proficiency in second language learners. Journal of Southwest Jiaotong University, 58(5), 601–610. https://doi.org/10.35741/issn.0258-2724.58.5.45
Bax, S. (n.d.). Text Inspector. The Centre of Research in English Language Learning and Assessment (CRELLA), University of Bedfordshire. https://textinspector.com/
Bordería, S. P., & Fischer, K. (2021). Using discourse segmentation to account for the polyfunctionality of discourse markers: The case of well. Journal of Pragmatics, 173, 101-118. https://doi.org/10.1016/j.pragma.2020.11.021
De Clercq, B., & Housen, A. (2017). A cross-linguistic perspective on syntactic complexity in L2 development: Syntactic elaboration and diversity. The Modern Language Journal, 101(2), 315–334. https://doi.org/10.1111/modl.12396
Derakhshan, A., & Ghiasvand, F. (2024). Is ChatGPT an evil or an angel for second language education and research? A phenomenographic study of research‐active EFL teachers’ perceptions. International Journal of Applied Linguistics, 34(4), 1246–1264. https://doi.org/10.1111/ijal.12561
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221–233. https://doi.org/10.1037/h0057532
Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied Linguistics, 21(3), 354–375. https://doi.org/10.1093/applin/21.3.354
Gregersen, T. (2023). Feedback matters: Thwarting the negative impact of language anxiety. Annual Review of Applied Linguistics, 43, 56–63. https://doi.org/10.1017/S026719052300003X
Gunning, R. (1952). The Technique of Clear Writing. McGraw-Hill.
Hayashi, K., & Sato, T. (2024). The effectiveness of ChatGPT in enhancing English language proficiency and reducing second language anxiety. WorldCALL 2023 Conference Proceedings, 201–208. https://doi.org/10.22492/issn.2759-1182.2023.23
Javier, D. R. C., & Moorhouse, B. L. (2024). Developing secondary school English language learners’ productive and critical use of ChatGPT. TESOL Journal, 15(2), Article e755. https://doi.org/10.1002/tesj.755
Joo, S. H. (2024). Generative AI as writing or speaking partners in L2 learning: Implications for learning-oriented assessments. Studies in Applied Linguistics & TESOL, 24(1), 54–59. https://doi.org/10.52214/salt.v24i1.12865
Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, Fog count and Flesch reading ease formula) for Navy enlisted personnel. (Research Report No. 56). Institute for Simulation and Training, University of Central Florida. https://stars.library.ucf.edu/istlibrary/56
Mizumoto, A., Shintani, N., Sasaki, M., & Teng, M. F. (2024). Testing the viability of ChatGPT as a companion in L2 writing accuracy assessment. Research Methods in Applied Linguistics, 3(2), Article 100116. https://doi.org/10.1016/j.rmal.2024.100116
Nebot, A. C. (2021). Prosodic modulation as a mark to express pragmatic values: The case of mitigation in Spanish. Journal of Pragmatics, 181, 196–208. https://doi.org/10.1016/j.pragma.2021.05.028
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518. https://doi.org/10.1093/applin/24.4.492
Read, J. (2000). Assessing Vocabulary. Cambridge University Press.
Simpson, R. C., Briggs, S. L., Ovens, J., & Swales, J. M. (2002). The Michigan Corpus of Academic Spoken English. The Regents of the University of Michigan.
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, Article 100752. https://doi.org/10.1016/j.asw.2023.100752
Ure, J. (1971). Lexical density and register differentiation. In G. E. Perren & J. L. M. Trim (Eds.), Applications of Linguistics (pp. 443–452). Cambridge University Press.
Yang, L., & Li, R. (2024). ChatGPT for L2 learning: Current status and implications. System, 124, Article 103351. https://doi.org/10.1016/j.system.2024.103351
Young, J. C., & Shishido, M. (2023). Investigating OpenAI’s ChatGPT potentials in generating Chatbot’s dialogue for English as a foreign language learning. International Journal of Advanced Computer Science and Applications, 14(6), 65–72. https://doi.org/10.14569/ijacsa.2023.0140607
Wei, L. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14, 1261955. https://doi.org/10.3389/fpsyg.2023.1261955
