The Multi-Sensory Training Benefit in Second Language Acquisition
Keywords:
second language acquisition, phonetics, cross-modal speech perception, multi-sensory training benefitAbstract
This article examines the role of multi-sensory input in the acquisition of non-native phonemes. First, it demonstrates that speech perception is not only auditory and that multi-sensory input such as audiovisual recordings of speech and tactile input from speech production aid the acquisition and comprehension of speech in one’s native language. Then, it examines studies that investigate the effectiveness of multimodal training compared to unimodal training when it comes to the identification of non-native phonemes. Based on the evidence presented, I argue that multi-sensory training aids the acquisition of non-native phonemes in some cases. However, when the additional modality provides no additional information, there seems to be no training benefit. There might be further limitations on the multi-sensory training benefit, such as whether it differs across native languages and can occur with other modalities than the audiovisual, but further research is needed to say anything conclusively in this area.
References
Bicevskis, K., D. Derrick & B. Gick. 2016. Visual-tactile integration in speech perception: Evidence for modality neutral speech primitives. The Journal of the Acoustical Society of America 140(5). 3531-3539. DOI: 10.1121/1.4965968.
Blumstein, S. & A. Cutler. 2003. Speech Perception. W. Frawley (ed.), International Encyclopedia of Linguistics. 2nd ed. Oxford: Oxford University Press. DOI: 10.1093/acref/9780195139778.001.0001.
Brown, V. A., M. Hedayati, A. Zanger, S. Mayn, L. Ray, N. Dillman-Hasso & J. F. Strand. 2018. What Accounts for Individual Differences in Susceptibility to the McGurk Effect? PLoS ONE 13(11). DOI: 10.1371/journal.pone.0207160
Colantoni, L., J. Steele & P. R. E. Neyra. 2015. Second Language Speech: Theory and Practice. Cambridge, United Kingdom: Cambridge University Press.
de Lima, A. L. & R. T. D. Nunes. 2015. Phonological profile of children with low vision from 6 to 9 years old at an institution for the blind in the city of Salvador. Revista CEFAC 17(5). 1490-1498. DOI: 10.1590/1982-0216201517521114.
Gentilucci, M. & L. Cattaneo. Automatic Audiovisual Integration in Speech Perception. 2005. Experimental Brain Research 167 (1). 66-75. DOI: 10.1007/s00221-005-0008-z.
Gick, B. & D. Derrick. 2009. Aero-tactile integration in speech perception. Nature 462(7272). 502-504. DOI: 10.1038/nature08572.
Grønnum, N. 1998. Danish. Journal of the International Phonetic Association 28(1-2). 99-105. DOI: 10.1017/S0025100300006290.
Hardison, D. M. 2003. Acquisition of second-language speech: Effects of visual cues, context, and talker variability. Applied Psycholinguistics 24. 495-522.
Hazan, V., A. Sennema, A. Faulkner, M. Ortega-Llebaria, M. Iba & Hyunsong Chung. 2006. The use of visual cues in the perception of non-native consonant contrasts. The Journal of the Acoustical Society of America 119(3). 1740-1751. DOI: doi.org/10.1121/1.2166611.
Hazan, V., A. Sennema, M. Iba & A. Faulkner. 2005. Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English. Speech Communication 47(3). 360-378. DOI: 10.1016/j.specom.2005.04.007.
Inceoglu, S. 2016. Effects of perceptual training on second language vowel perception and production. Applied Psycholinguistics 37(5). 1175-1199. DOI: 10.1017/S0142716415000533.
Lee, H., J. Lee, H. Hwang & J. Y. Lim. 2018. Effects of audiovisual perceptual training with corrective feedback on the perception and production of English sounds by Korean learners. The Journal of the Acoustical Society of America 144(3). 1867-1867. DOI: 10.1121/1.5068210.
McGurk, H. & J. MacDonald. 1976. Hearing lips and seeing voices. Nature 264(5588). 746-748. DOI: 10.1038/264746a0.
Mills, A. E. 1983. Acquisition of speech sounds in the visually-handicapped child. Mills, A.E. (ed.), Language acquisition in the blind child: Normal and deficient, 45-56. San Diego: College Hill.
Montgomery, A. A., B. E. Walden, D. M. Schwartz & R. A. Prosek. 1984. Training Auditory-Visual Speech Reception in Adults with Moderate Sensorineural Hearing Loss. Ear and Hearing 5. 30-36.
Munhall, K. G., P. Gribble, L. Sacco & M. Ward. 1996. Temporal constraints on the McGurk effect. Perception & Psychophysics 58(3). 351-62. DOI: 10.3758/bf03206811.
Rosenblum, L. D. 2019. Does Sensory Modality Matter? Not for Speech Perception. Acoustical Society of America 15(3). 47-54.
Rosenblum, L.D., R. M. Miller & K. Sanchez. 2007. Lip-Read Me Now, Hear Me Better Later: Cross-Modal Transfer of Talker-Familiarity Effects. Psychological Science 18(5). 392-396. DOI: 10.1111/j.1467-9280.2007.01911.x.
Sekiyama, K., D. Burnham, H. Tam & D. Erdener. 2003. Auditory-visual speech perception development in Japanese and English talkers. Proceedings of the International Conference on Auditory-Visual Speech Processing, St. Jorioz, France. 61-66. https://www.researchgate.net/publication/237903326_Auditory-visual_speech_perception_development_in_Japanese_and_English_talkers (Accessed 16 Apr 2023).
Sekiyama, K. & Y. Tohkura. 1991. McGurk Effect in Non-English Listeners: Few Visual Effects for Japanese Subjects Hearing Japanese Syllables of High Auditory Intelligibility. The Journal of the Acoustical Society of America 90(4). 1797-1805. DOI: 10.1121/1.401660.
Schall, S. & K. von Kriegstein. 2014. Functional connectivity between face-movement and speech-intelligibility areas during auditory-only speech perception. PLoS ONE 9. DOI: 10.1371/journal.pone.0086325.
Schwartz, J., F. Berthommier & C. Savariaux. 2004. Seeing to Hear Better: Evidence for Early Audio-Visual Interactions in Speech Identification. Cognition 93(2). B69-B78. DOI: 10.1016/j.cognition.2004.01.006.
Van Engen, K. J., A. Dey, M. S. Sommers & J. E. Peelle. 2022. Audiovisual speech perception: Moving beyond McGurk. The Journal of the Acoustical Society of America 152(6). 3216-3225. DOI: 10.1121/10.0015262.
Wang, Y., D. M. Behne & H. Jiang. 2008. Linguistic experience and audio-visual perception of non-native fricatives. The Journal of the Acoustical Society of America 124(3). 1716-1726. DOI: 10.1121/1.2956483.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Author and Journal of Language Works
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The author/the authors hold the rigths to articles presented in the journal. The author/the authors are granted the right to reproduce their article as they see fit, if they mention LWorks as the original publisher of the article.
