McGurk effect

The McGurk effect is a phenomenon that occurs when the brain processes spoken language that conflicts with visual stimulus (e.g. a video that’s been overdubbed with an entirely separate audio track). Basically, when given audio that includes one consonantal phoneme (e.g. the /b/ in /ba/), combined with visuals of a different phoneme being mouthed (like the /g/ in /ga/), the brain tries to combine the two contradictory sources of information and what the individual hears is a different phoneme again (the /d/ in /da/).

Different vowels affect how well the illusion works. Before /a/, the effect is strong. Before /i/, not so much. Before /u/, the effect barely occurs. It’s thought that this might be because the mouth has to be held in a more specific way to produce /i/ or /u/ than /a/, so the brain has more visual information with which to dispell the illusion.

The illusion is very robust, and occurs even when people know full well that they’re watching a demonstration of the illusion. However, it seems that people with certain forms of neurodivergence are more resistant to its effect. For example, autistic children have been shown to be more resistant than allistic children, although the difference gets smaller comparing adults. People with dyslexia, aphasia and language-learning disabilities have also been shown to have a reduced McGurk effect.

Interestingly, the strength of the effect also varies depending on what language is being spoken. The McGurk effect has been shown to be relatively robust for English, German, Dutch, Spanish, Italian and Turkish, but it is markedly less so for Japanese and Chinese.

A number of other factors can also impact the likelihood of experiencing the effect, like semantic congruence (your brain is more likely to put together data to “hear” a word that exists than a nonsense word), level of distraction (the effect is greatly reduced if you’re distracted), familiarity to the speaker (you are more likely to succumb to the effect if watching a video of a stranger) and even gender of the listener (women are more likely to experience the illusion than men).

See Also / References