Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
New system turns camera into an optical directional microphone

New system turns camera into an optical directional microphone

[10:09 Sat,25.June 2022   by ]    

Researchers at Carnegie Mellon University have developed a camera system that can detect sound vibrations with such precision and detail that it is able to reconstruct the sound of a single instrument in the midst of an orchestra using only video, thus functioning as a quasi-optical directional microphone that can pick up a specific sound source even from some distance.

Dual-Shutter-Muster




Even the most powerful and well-directed microphones cannot eliminate background noise, ambient noise, and the effects of acoustics such as reverberation when recording audio. The novel system, on the other hand, uses two cameras capable of capturing video at 60 and 130 fps, respectively, and a low-power laser (with the light intensity of a standard laser point) to capture high-velocity, low-amplitude surface vibrations that are invisible to the human eye. These vibrations can be used to reconstruct sound, capturing isolated audio signals without inference or a microphone.



The team conducted several successful demos to test the system&s function in capturing vibrations and the quality of sound reconstruction. A primary instrument can be used as a sound source as well as an object that vibrates only indirectly. Thus, isolated audio data were successfully recorded from individual guitars playing simultaneously as well as from individual loudspeakers playing different music simultaneously. But the sound of a speaker could also be analyzed indirectly by means of vibrations from a tuning fork or a chip bag located near the speaker.

The new system is a significant improvement over previous attempts to capture sound using computer vision-so it now uses two ordinary cameras (one with a rolling shutter, one with a global shutter) that cost only a fraction of the high-speed versions used in previous research, but now deliver higher recording quality at the same time.

How does the optical microphone work?


The system works by analyzing the differences in the speckle patterns of images captured with a rolling shutter and a global shutter. An algorithm calculates the differences in these patterns of the two video streams and converts these differences into oscillations to reconstruct the sound.

A speckle pattern refers to the way coherent light behaves in space after reflecting off a rough surface. The team creates the speckle pattern by pointing a laser at the surface of the object producing the vibrations, such as a guitar body. This speckle pattern changes as the surface vibrates. A rolling shutter captures an image by scanning it quickly, usually from top to bottom, creating the image by stacking a row of pixels. A global shutter captures an image in a single instant.

Dual-Shutter



What it can do

.
In the future, the team&s new optical vibration detection system could allow sound engineers, for example, to monitor the music of individual instruments without disturbing the rest of the ensemble in order to fine-tune the overall mix. Or manufacturers could use the system to monitor the vibrations of individual machines on a factory floor to detect early signs that maintenance is needed. But of course, the new system could also be used to record conversations undetected from a distance, for example, as is already possible with special laser-based systems.

The research paper titled "Dual-Shutter Optical Vibration Sensing" can be downloaded here.

Link more infos at bei imaging.cs.cmu.edu

deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon

  

  Vorige News lesen Nächste News lesen 
bildBlackmagic Camera 7.9 Update bringt besseren Autofokus und Bildstabilisation per Gyrodaten bildDaVinci Resolve 18 Beta 5 bringt KI-Stabilizer für Gyro-Metadaten und 10 Bit Vorschau


related news:1E0Rode introduces compact VideoMicro II for cameras and smartphones for 92,- Euro 30.November 2022
Meta Encodec uses AI to compress audio files significantly more than MP3 21.November 2022
RØDE NT-USB+: Professional USB condenser microphone with integrated DSP 18.November 2022
Hollyland Lark C1: Affordable wireless two-channel microphone system especially for smartphones 7.November 2022
Sony C-80 studio microphone 8.October 2022
Blackmagic slashes price of ATEM Mini Pro live mixer and streamer 7.October 2022
OM System LS-P5: compact intelligent audio recorder for filmmakers 24.August 2022
alle Newsmeldungen zum Thema Sound
1E0Rode introduces compact VideoMicro II for cameras and smartphones for 92,- Euro 30.November 2022
UPDATE 3: Cyber Monday and other discount offers from Dedolight, Sennheiser, Asus, Samyang, Lowepro, 27.November 2022
RØDE NT-USB+: Professional USB condenser microphone with integrated DSP 18.November 2022
Hollyland Lark C1: Affordable wireless two-channel microphone system especially for smartphones 7.November 2022
Sony C-80 studio microphone 8.October 2022
Azden SGM-250H: professional shotgun microphone 4.August 2022
Sony introduces new compact shotgun microphone ECM-G1 14.July 2022
alle Newsmeldungen zum Thema Microphones
1E0Filmic Pro 7: Professional camera app for Andoid and iOS with new features and subscription model 30.November 2022
TTArtisan C 35mm f/0.95 super fast manual S35/APS-C lens for 295,- Euro 28.November 2022
Survey of 1,000 photographers (pros and amateurs): Which camera do you use? 21.November 2022
Sony Wintercashback and Trade-In 2022: Save up to 400 Euro when buying camera and lens 12.November 2022
PortKeys PT6: Affordable 5.2 7.November 2022
For Nikon Z: MC-N10 remote control handle makes filming more ergonomic 3.November 2022
DJI Mavic 3 Classic: Significantly cheaper than Mavic 3, but with the same main camera 2.November 2022
alle Newsmeldungen zum Thema Camcorders


[nach oben]
















Archiv Newsmeldungen

2022

December - November - October - September - August - July - June - May - April - March - February - January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon



last update : 1.Dezember 2022 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version