Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· KI· Reviews· Streaming
/// News
New system turns camera into an optical directional microphone

New system turns camera into an optical directional microphone

[10:09 Sat,25.June 2022   by ]    

Researchers at Carnegie Mellon University have developed a camera system that can detect sound vibrations with such precision and detail that it is able to reconstruct the sound of a single instrument in the midst of an orchestra using only video, thus functioning as a quasi-optical directional microphone that can pick up a specific sound source even from some distance.

Dual-Shutter-Muster




Even the most powerful and well-directed microphones cannot eliminate background noise, ambient noise, and the effects of acoustics such as reverberation when recording audio. The novel system, on the other hand, uses two cameras capable of capturing video at 60 and 130 fps, respectively, and a low-power laser (with the light intensity of a standard laser point) to capture high-velocity, low-amplitude surface vibrations that are invisible to the human eye. These vibrations can be used to reconstruct sound, capturing isolated audio signals without inference or a microphone.



The team conducted several successful demos to test the system&s function in capturing vibrations and the quality of sound reconstruction. A primary instrument can be used as a sound source as well as an object that vibrates only indirectly. Thus, isolated audio data were successfully recorded from individual guitars playing simultaneously as well as from individual loudspeakers playing different music simultaneously. But the sound of a speaker could also be analyzed indirectly by means of vibrations from a tuning fork or a chip bag located near the speaker.

The new system is a significant improvement over previous attempts to capture sound using computer vision-so it now uses two ordinary cameras (one with a rolling shutter, one with a global shutter) that cost only a fraction of the high-speed versions used in previous research, but now deliver higher recording quality at the same time.

How does the optical microphone work?


The system works by analyzing the differences in the speckle patterns of images captured with a rolling shutter and a global shutter. An algorithm calculates the differences in these patterns of the two video streams and converts these differences into oscillations to reconstruct the sound.

A speckle pattern refers to the way coherent light behaves in space after reflecting off a rough surface. The team creates the speckle pattern by pointing a laser at the surface of the object producing the vibrations, such as a guitar body. This speckle pattern changes as the surface vibrates. A rolling shutter captures an image by scanning it quickly, usually from top to bottom, creating the image by stacking a row of pixels. A global shutter captures an image in a single instant.

Dual-Shutter



What it can do

.
In the future, the team&s new optical vibration detection system could allow sound engineers, for example, to monitor the music of individual instruments without disturbing the rest of the ensemble in order to fine-tune the overall mix. Or manufacturers could use the system to monitor the vibrations of individual machines on a factory floor to detect early signs that maintenance is needed. But of course, the new system could also be used to record conversations undetected from a distance, for example, as is already possible with special laser-based systems.

The research paper titled "Dual-Shutter Optical Vibration Sensing" can be downloaded here.

Link more infos at bei imaging.cs.cmu.edu

deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon

  

  Vorige News lesen Nächste News lesen 
bildBlackmagic Camera 7.9 Update bringt besseren Autofokus und Bildstabilisation per Gyrodaten bildDaVinci Resolve 18 Beta 5 bringt KI-Stabilizer für Gyro-Metadaten und 10 Bit Vorschau


related news:1E0From text to audio: Stable Audio generates music and sound effects with AI 18.September 2023
Sennheiser introduces MKH 8030 - HF condenser microphone with figure-of-eight pattern 16.September 2023
Grass Valley announces EDIUS 11 with KI-based functions, dedicated audio editor and more 15.September 2023
If you could only choose 1 microphone: shotgun, compact condenser or lavalier? 29.August 2023
YouTube integrates automatic audio dubbing through AI 31.July 2023
Sony Shotgun Microphone ECM-M1: Per selectable polar pattern 8 mics in one? 16.July 2023
RØDE PodMic USB: Studio microphone with XLR and USB 25.May 2023
alle Newsmeldungen zum Thema Sound
1E0Sennheiser introduces MKH 8030 - HF condenser microphone with figure-of-eight pattern 16.September 2023
If you could only choose 1 microphone: shotgun, compact condenser or lavalier? 29.August 2023
Rode Wireless Pro 2-channel radio link with 32-bit float recording, timecode sync and more unveiled 23.August 2023
Rode: New charging case for the Wireless GO II - TX modules for ME and GO II available separately 1.August 2023
Sony Shotgun Microphone ECM-M1: Per selectable polar pattern 8 mics in one? 16.July 2023
RØDE PodMic USB: Studio microphone with XLR and USB 25.May 2023
Deity PR-2 Pocket Audio Recorder: 32-bit Float Recording and Preamplifier 12.May 2023
alle Newsmeldungen zum Thema Microphones
1E0RED's high-end cine camera series V-Raptor gets face detection autofocus 29.September 2023
IMAX blockbuster The Creator shot with 5,000 Euro camera - Sony'amp;s FX3 28.September 2023
Nikon Z f retro-look full-frame DSLM (FM2) unveiled for 2,499 Euro: 24MP sensor, 4K 10-bit 60p ... 20.September 2023
Videoclip: Blackmagic Cinema Camera 6K explained 16.September 2023
Interview: Blackmagic iPhone Camera App and Cloud explained 16.September 2023
Blackmagic Micro Studio Camera 4K G2 - updated and available soon 16.September 2023
Blackmagic launches free professional camera app for iPhone: Camera App 14.September 2023
alle Newsmeldungen zum Thema Camcorders


[nach oben]
















Archiv Newsmeldungen

2023

October - September - August - July - June - May - April - March - February - January

2022
December - November - October - September - August - July - June - May - April - March - February - January

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon



last update : 1.Oktober 2023 - 15:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version