A Cross-linguistic Examination on the McGurk Effect in Different Developmental States

Abstract

McGurk and MacDonald (1976) found the visual illusion in audiovisual speech perception, called “the McGurk effect”, where a face articulating /ga/ was presented synchronously with an auditory /ba/ syllable and subjects reported that they had perceived it as the syllable /da/, which compromised the conflicting visual and auditory cues. Although the visual influence in audiovisual speech perception was robustly demonstrated in several aspects during the last four decades, the consensus regarding the McGurk effect has not been reached in two aspects: (1) why does the magnitude of the McGurk effect differ across languages; (2) why does the development of audiovisual speech perception represent a U- shaped pattern, i.e., the visual influence is strong in infants, adolescents and adults while weak in children especially 4- to 6- year-old preschool children. The current thesis, therefore, focusing on these two aspects, provides two hypotheses to tackle each aspect respectively. The first hypothesis, called the visual speech intelligibility hypothesis, suggests that speakers native to languages with a higher degree of visual intelligibility adopt more visual information in their audiovisual speech perception and therefore demonstrate a stronger McGurk effect. The second hypothesis suggests that early literacy training, by accelerating the development of phonological awareness, facilitates children to adopt phonetic visual information in audiovisual speech perception and this acceleration is related to the orthography. The two hypotheses, with the first targeting on cross-linguistic differences and the second on differences across developmental states, collectively provides a holistic framework for the cross-linguistic differences on the developmental trajectories of audiovisual speech perception from childhood to adulthood.