When sound hits an object, it causes small vibrations of the object’s surface. This is why standard microphones work, but it’s also the principle behind this video, which explores using video as a source for resynthesizing sound.
The video, via Abe Davis’s Research, demonstrates how high-speed video can be used to capture micro-vibrations in objects filmed, which can be used as a visual microphone. Those visual vibrations can be used to, at least partially, recover the sound that produced them. This lets everyday objects—a glass of water, a potted plant, a box of tissues, or a bag of chips—be used as ‘visual microphones’.
The video demonstrates capturing sounds from high-speed footage of a variety of objects with different properties, and explores some of the factors that affect our ability to visually recover sound. The video also explores how to use regular consumer cameras to recover audio from standard frame-rate videos.
Additional information on this technique can be found and the project’s web page.