Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
New system turns camera into an optical directional microphone

New system turns camera into an optical directional microphone

[10:09 Sat,25.June 2022   by ]    

Researchers at Carnegie Mellon University have developed a camera system that can detect sound vibrations with such precision and detail that it is able to reconstruct the sound of a single instrument in the midst of an orchestra using only video, thus functioning as a quasi-optical directional microphone that can pick up a specific sound source even from some distance.

Dual-Shutter-Muster




Even the most powerful and well-directed microphones cannot eliminate background noise, ambient noise, and the effects of acoustics such as reverberation when recording audio. The novel system, on the other hand, uses two cameras capable of capturing video at 60 and 130 fps, respectively, and a low-power laser (with the light intensity of a standard laser point) to capture high-velocity, low-amplitude surface vibrations that are invisible to the human eye. These vibrations can be used to reconstruct sound, capturing isolated audio signals without inference or a microphone.



The team conducted several successful demos to test the system&s function in capturing vibrations and the quality of sound reconstruction. A primary instrument can be used as a sound source as well as an object that vibrates only indirectly. Thus, isolated audio data were successfully recorded from individual guitars playing simultaneously as well as from individual loudspeakers playing different music simultaneously. But the sound of a speaker could also be analyzed indirectly by means of vibrations from a tuning fork or a chip bag located near the speaker.

The new system is a significant improvement over previous attempts to capture sound using computer vision-so it now uses two ordinary cameras (one with a rolling shutter, one with a global shutter) that cost only a fraction of the high-speed versions used in previous research, but now deliver higher recording quality at the same time.

How does the optical microphone work?


The system works by analyzing the differences in the speckle patterns of images captured with a rolling shutter and a global shutter. An algorithm calculates the differences in these patterns of the two video streams and converts these differences into oscillations to reconstruct the sound.

A speckle pattern refers to the way coherent light behaves in space after reflecting off a rough surface. The team creates the speckle pattern by pointing a laser at the surface of the object producing the vibrations, such as a guitar body. This speckle pattern changes as the surface vibrates. A rolling shutter captures an image by scanning it quickly, usually from top to bottom, creating the image by stacking a row of pixels. A global shutter captures an image in a single instant.

Dual-Shutter



What it can do

.
In the future, the team&s new optical vibration detection system could allow sound engineers, for example, to monitor the music of individual instruments without disturbing the rest of the ensemble in order to fine-tune the overall mix. Or manufacturers could use the system to monitor the vibrations of individual machines on a factory floor to detect early signs that maintenance is needed. But of course, the new system could also be used to record conversations undetected from a distance, for example, as is already possible with special laser-based systems.

The research paper titled "Dual-Shutter Optical Vibration Sensing" can be downloaded here.

Link more infos at bei imaging.cs.cmu.edu

deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon

  

  Vorige News lesen Nächste News lesen 
bildBlackmagic Camera 7.9 Update bringt besseren Autofokus und Bildstabilisation per Gyrodaten bildDaVinci Resolve 18 Beta 5 bringt KI-Stabilizer für Gyro-Metadaten und 10 Bit Vorschau


related news:1E0Azden SGM-250H: professional shotgun microphone 4.August 2022
Bluetooth LE Audio brings much better sound quality and finally a Brodcast mode 16.July 2022
Sony introduces new compact shotgun microphone ECM-G1 14.July 2022
Hollyland Lark M1: Affordable two-channel wireless clip-on microphone system 6.July 2022
Blackmagic Pocket Cinema Camera 6K G2 - Bigger battery, 2x XLR and foldable display 23.June 2022
Sony ECM-B10: Compact shotgun microphone with selectable polar pattern 17.June 2022
Zhiyun Weebill 3: Professional one-hand gimbal for DSLRs and DSLMs integrated light and mic 16.June 2022
alle Newsmeldungen zum Thema Sound
1E0Azden SGM-250H: professional shotgun microphone 4.August 2022
Sony introduces new compact shotgun microphone ECM-G1 14.July 2022
Hollyland Lark M1: Affordable two-channel wireless clip-on microphone system 6.July 2022
Sony ECM-B10: Compact shotgun microphone with selectable polar pattern 17.June 2022
DJI Mic: 2-channel clip-on wireless microphone system with two transmitters 5.April 2022
Rode NTH-100: First professional studio headphones for audio monitoring, filmmaking and more. 29.March 2022
Deity VO-7U: Dynamic USB Microphone for Podcasters 18.February 2022
alle Newsmeldungen zum Thema Microphones
1E0Atomos Connect now supports Frame.io Camera-to-Cloud 12.August 2022
Blackmagic Kamera 7.9.1 Update 29.July 2022
Canon EOS R5: extended recording times with firmware update v1.60 22.July 2022
Nikon Z 30: Compact 4Kp30 APS-C camera for vlogging 29.June 2022
Tilta: Camera Cage for Canon EOS R5C incl. battery solution for 8K 50/60p RAW 27.June 2022
Blackmagic Camera 7.9 update brings new camera GUI and image stabilization via gyro data 24.June 2022
Blackmagic Pocket Cinema Camera 6K G2 - Bigger battery, 2x XLR and foldable display 23.June 2022
alle Newsmeldungen zum Thema Camcorders


[nach oben]
















Archiv Newsmeldungen

2022

August - July - June - May - April - March - February - January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Forschungsprojekt macht Kamera zu optischem Richtmikrofon



last update : 12.August 2022 - 12:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version