[12:44 Mon,16.May 2022 by Thomas Richter] |
In order to record recognizable videos despite poor lighting conditions, the gain is increased - this makes each pixel more light-sensitive. However, this also increases the noise in each image: the brighter the image is made via gain, the more noise it also produces. With particularly sensitive CMOS cameras, it is currently possible to record videos at night only in moonlight (i.e. an illuminance of only 0.05-0.3 lux), but these are relatively noisy. ;)
A team from Intel Labs and the University of California Berkeley, in a new research paper nicely titled "Dancing under the stars: video denoising in starlight", has now presented a new AI method for denoising videos recorded under extremely low illumination of fractions of a lux. ;) The new deep-learning denoising method is demonstrated using 5-10 fps video recordings with a special RGB plus NIR (near infrared light) camera illuminated only by starlight at 0.6 millilux. The end result is photo-realistic footage with virtually no image noise or other image defects normally associated with low-light conditions. The algorithm was tested with several challenging scenes with lots of motion, where people dance illuminated only by the light of the Milky Way, while a meteor shower rains down from above. The new algorithm was trained using video footage that was artificially noisy using a special method as well as real noisy video. The artificially noisy videos automatically produced reference material which could be used to estimate how well the denoising works and could be improved based on this (otherwise it is very difficult to record original material together with an otherwise identical noisy version in parallel or successively for moving images). ;) The new algorithm also performs very well in comparison with other methods: Inserted sensor.The extremely fast  Canon LI3030 SAI RGB-NIR sensor with a resolution of 2,160 x 1,280 19µm pixels, analog 16-channel output and increased quantum efficiency in the near infrared rangeg (NIR) was used. This uses a Bayer pattern consisting of red, green, blue (RGB) and NIR channels (800-950nm). Each RGB channel has an additional transmittance peak that overlaps with the NIR channel to increase light throughput at night. During daylight, the NIR channel can be subtracted from each RGB channel Channel Limitations.However, the method chosen by the researchers has its limitations. First, the noise generator used for training currently mimics only a single gain setting (in this case, the highest gain). In addition, the colors of the denoised night videos tend to be dull, since a large portion of the image comes from the monochrome near-infrared spectral range. However, special DeepLearning methods could be used to colorize the images correctly in the future. Translated with www.DeepL.com/Translator (free version)  more infos at bei kristinamonakhova.comdeutsche Version dieser Seite: Nur Sternenlicht als Beleuchtung: Neuer KI-Algorithmus entrauscht Videos perfekt |
 |
|
|