[10:36 Sat,13.December 2025 by Thomas Richter] |
Video AIs have improved at breakneck speed and now generate videos that are barely distinguishable from reality. To achieve this, they must follow all the complex laws of physics in the depiction of scenes: objects in the foreground obscure objects in the background, some objects are transparent, light refraction, shadow casting, movement of liquids, etc. – not to mention the peculiarities of the movement of animate objects or human facial expressions.
;) A question that arises is: do AIs understand the world, or do they merely extrapolate from the enormous training material? This was already a topic when OpenAI introduced the first realistic video AI, Sora, 1.5 years ago, when OpenAI concurrently published the article  Video generation models as world simulators, which concluded with the sentence: "We believe that Sora&s current capabilities show that continued scaling of video models is a promising path toward developing powerful simulators of the physical and digital world, and the objects, animals, and people within them." ;) The different test scenarios A team of researchers from Google&s DeepMind put this belief to the test almost a year ago, investigating whether modern video AIs truly "understand" physical laws. To do this, they developed a benchmark called Physics IQ, which can only be solved through a deep understanding of various physical principles such as fluid dynamics, optics, solid mechanics, magnetism, and thermodynamics. ;) Test procedure Each test shows the beginning of an event using a real video and then asks a video AI to predict the next few seconds. This prediction is then compared with the actual course of events – through motion analysis that checks where, when, and how strongly things move. Depending on how closely the predictions match reality, a Physics IQ score is calculated. The results show that the physical understanding of all investigated video AIs (such as Sora, Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet) is severely limited and bears no relation to visual realism. For example, while the videos generated by Sora are the most difficult to distinguish from real videos, the model&s physics score is low – which shows that realism and physical understanding are not correlated. ;) Physics IQ score of the different video AI models However, some test scenarios were solved successfully and predicted correctly by some models. This suggests that learning certain physical principles solely through observation might be possible – but significant challenges remain. The researchers anticipate rapid progress in the near future, but their work demonstrates that visual realism does not imply physical understanding and thus an internal world model.  more infos at bei arxiv.orgdeutsche Version dieser Seite: Verstehen Video-KIs die Welt? Physik-IQ enthüllt Grenzen der Modelle |
 |
|
|