Manipulation of Sounds through Sight - an Extension of the McGurk Effect Alex Liu

Progress Journal:

Elective reading journal entry:

I did two elective readings, listening to both the Radiolab and the TED Talk by Issac Lidsky. The TED Talk really resonated me, as it was a guy who went blind at the age of 25 presenting on the senses, especially sight. While sight may be our dominant sense, taking up around 30% of our visual cortex, we do not absolutely need it, and losing sight can in fact be a blessing. For Lidsky, it taught him that he was in control of his own reality, and that blindness itself would not inhibit what he wanted to do. I learned that what we see is a personal, virtual reality, and not necessarily objective truth. Sight is altered by our pre-existing knowledge, memories, and emotions, and for example, a landmark appears farther away when wearing a heavy backpack. We can learn to alter our sight, and similarly we can learn to create our own reality, something that being blind taught Lidsky.


The documentation contains our script, our experiment, and the video we used to make the experiment.


Mack: For our project, we decided to do an altered version of the McGurk effect. We started out the project with the question, “To what degree are sense perceptions altered by our expectations”. We hope that this project shows how sense perceptions can be altered to a high degree because of different sense biases we hold. During this kahoot, simply watch the video, and choose the choice that best represents what sound you heard. Do not discuss the answers with those around you and stay focused on the screen throughout the videos.

Valeria: So when we first started this project, we hypothesized that Sense Perceptions were easily altered by expectations. After playing around with many different vowel and consonant combinations, we decided that Sense Perceptions were able to be manipulated, but it was not easy. We settled on around 10 combinations that were effective most of the time, and decided to record video clips of Paschal saying these sounds and then quiz the class to test our hypothesis.

Alex: As the results of the quiz indicate, our sense perceptions are not always guaranteed to be accurate and often our senses confuse each other. For example, after hearing the sound “bah” with your eyes closed, you could most likely realize that the sound bah was being made. However, when your eyes interpret the sensations of the mouth making the motion “Fah” as the sound of “Bah” is played, your eyes begin to weigh in, and sometimes overrule your ears. This is key in understanding in how our Sense Perceptions allow us to perceive the world.

Mack: The McGurk effect was originally discovered in 1976 when researchers were attempting to study infants’ change in perception of language based on the developmental stage they were in. This effect takes place “because of the brain's effort to provide the consciousness with its best guess about the incoming information.”

Valeria: This leads back to Sense Perception. We believe our brains to be absolute and knowing regarding interpreting the world around us, however it is clear that our brain is simply giving us its “best guess” when given sensation to analyze.

Mack: Our biases play a large role in our interpretation of the environment around us. We use previous knowledge so that we are able to fill the gaps in our surroundings without much brain power dedicated to analyzing those sensations. However, these biases often impact how we perceive the world. One effect that makes the McGurk effect a lot stronger is the expectation of a certain word in a sentence. For example if it is a sentence you hear often, “Hello, how are you”, if a different audio clip was played that said “Hello, how are new” or “Hello, how are few”, and yet the visual clip showed someone mouthing you, the McGurk effect would be very strong. We use our biases to make sense of the world. If we never had any biases to operate off of, the world would be consistently and never endingly complex.

Alex: We started with the question, “To what degree are our sense perceptions shaped by expectations”. We hope that through this experiment, you are able to tell that Sense Perceptions, especially when dealing with multi-modal sensations (Using several senses to interpret information, are unreliable and subject to our biases gained through prior experiences.

Mack: Now, from this experiment we can gain many insights into something deeper than simply our sense-perceptions. The fact that our biases so easily, and subconsciously, affect our interpretation of the world suggests that our biases may entirely shift the way we gain knowledge, and therefore impact our ability to truly understand the world. In fact, many of the Theories of Reality discussed in Chapter 5 are placed in danger because of this experiment. For example: Common Sense Realism. This theory operates around the fact that the way we perceive the world mirrors the way the world actually is. And Phenomenalism, based on empiricism, is also threatened. If we can not trust our senses, which is clearly shown in this experiment, how can we trust that we can only verify something if we have witnessed it?

Valeria: The McGurk effect disproved several theories, and leaves us with many questions, to end the presentation, consider this: “If we can not verify the reality of our world through our senses, how can we verify anything?”

Our experiment:

We made a Kahoot with 7 questions, and each question contained a video clip. In each video clip, Paschalis would mouth a sound, while playing the audio clip of him saying a different sound. We had the class choose one of four choices that most closely resembled what sound they heard. The video clips are drawn from the video below.

Results of our experiment:

Out of the 112 (7 questions and 16 people) choices that the class made, only 40 were correct (they chose the correct sound being played), a 36% accuracy. Not accounting for one problem, where 13 out of 16 people got it correct (the sounds were likely not close enough to each other), the class accuracy on the other problems was only 27 out of 96, or 28%. Thus, the results of this experiment demonstrate that our sense perception is easily fooled by our expectations, as the class was unable to accurately distinguish the correct sounds due to conflicting mouth movements.

Extension Proposal:

I propose the inclusion of the following essential question to the list: To what extent does the role of sense perception vary across different Areas of Knowledge? I think that this is a very important question that strongly pertains to the Way of Knowing sense perception. For example, in art, each person perceives a particular work of art differently and subjectively, while in science the goal is for sense perception to be uniform, as if many people perceived a science experiment differently, then no conclusion could be drawn from it. This question can easily be explored and has a wide range of possibilities, since one can choose any Areas of Knowledge, so I believe that it belongs in the list of essential questions for sense perception.

Made with Adobe Slate

Make your words and images move.

Get Slate

Report Abuse

If you feel that this video content violates the Adobe Terms of Use, you may report this content by filling out this quick form.

To report a Copyright Violation, please follow Section 17 in the Terms of Use.