Track-Anything: New AI tool automatically masks and tracks objects in videos

[15:59 Thu,27.April 2023 by Thomas Richter]

Roto Brush in After Effects Adé - AI Tools Track-Anything automatically detects, masks and tracks objects in video. As hoped, Meta&s freely available Segment Anything (SAM) algorithm, which detects objects in images and automatically separates them as individual masks, has led to a veritable explosion of tools based on it. Of particular interest for video is the possibility to automatically crop moving objects from video to edit or even remove them from a video.

Track Anything

A new AI tool

Track-Anything from a team at the Visual Intelligence & Perception Lab department at China&s Southern University of Science and Technology in Shenzen does just that. It is a flexible and interactive tool for tracking and segmenting video objects. It is based on Segment Anything and allows users to select objects to be tracked and masked with just a few clicks. During tracking, users can flexibly change the objects to be tracked or even correct the area to be analyzed if there are tracking problems.

Track-Anything not only tracks objects from frame to frame, but also recognizes them across scenes, as can be seen nicely in the following demo video based on a scene from "Avengers Civil War":

Multiple objects can also be masked and tracked at the same time - furthermore, inpainting allows "Track anything" to remove a tracked object from the video:

There is also a

interactive demo on Hugginface with the help of which you can create your own video or one of the sample clips. However, since there is often too weig GPU power available, using it on your own PC or cloud space - assuming a strong GPU - is more reliable.

more infos at bei github.com

deutsche Version dieser Seite: Track-Anything: Neues KI-Tool maskiert und trackt Objekte in Videos automatisch