[10:52 Thu,1.August 2024 by Thomas Richter] |
Detecting moving objects in videos, automatically masking them, and tracking them across entire video sequences is the foundation for many interesting video effects. Meta&s Segment Anything Algorithm does exactly that—with very high quality. Objects can be selected by the user in real time, and they are then automatically tracked and masked in all subsequent frames of a video. Effects can then be specifically applied to these objects (or the background). The new version can also track objects across multiple settings or even when they are temporarily obscured by another object. ![]()
A practical use case in video editing is, for example, blurring license plates or faces—SAM2 can pixelate selected objects with minimal effort across multiple scenes. Combined with generative video AIs, there are also many interesting applications for SAM2 for very specific effects. The algorithm can, of course, also be applied to individual images for image editing.
The previous version was also used for scientific purposes, such as in marine science to segment sonar images and analyze coral reefs, in satellite image analysis for disaster relief, and in the medical field for segmenting cell images and assisting in the detection of skin cancer. The new version works in real time and can therefore also be used in live video streams, such as in sports broadcasts. ![]() Fortunately, the improved Segment Anything Model 2 (SAM 2)—like the first version released in April last year—is available under the free Apache 2.0 license ![]()
SAM 2 surpasses other algorithms for interactive video segmentation in terms of object recognition accuracy and tracking speed. Additionally, it can recognize and mask any object in any video or image (often described as zero-shot generalization), meaning it works even with completely new, previously unseen, and thus untrained objects. Try it yourselfWith the ![]() Many users have already done this and posted their results online:
![]() deutsche Version dieser Seite: Meta Segment Anything 2 - kostenloses Tool maskiert interaktiv Objekte in Videos |
![]() |