Google DeepMind Genie 3 - generate interactive worlds in real time

[20:27 Tue,5.August 2025 by Thomas Richter]

Google&s DeepMind has introduced Genie 3, an interactive world generator that creates worlds via prompt, which can then be traversed in real-time - much like the famous Holodeck from Star Trek. This is revolutionary in several respects, as Google has solved several problems in the third generation of its World Building Model.

Consistent Interactive Worlds

The generated worlds are now relatively consistent, the model has a "memory," meaning that the images are constantly being generated live, but the world is not constantly completely new. Instead, an object or place once visited is still identical on a second visit - which is fundamentally important for the feeling of exploring another world. Genie 3 is not perfect (the memory only covers the last minute, the entire environment at least several minutes), but already better than previous models.

In addition to the whole world, specific objects can also be prompted, such as a gorilla in a red tailcoat:

The quality of the real-time generated images is also impressive - the live video has a resolution of 720p (1,280 x 720 pixels) at a frame rate of 20-24 frames per second. Even with fast movement, no errors can be detected in the demo videos - they actually give the feeling of wandering in another world, because the generated worlds look seamless in all directions.

Genie 3 "understands" the world including all physical laws, just like video AIs like Veo 3 or Sora, and can thus simulate spatiality deceptively and "knows" that water reflects light and creates waves when touched, how waves behave when they hit solid objects, how objects behave upon collision depending on their relative weight and material, such as here in the collision with one of the lanterns:

Besides real worlds, fantastic ones can just as easily be generated and explored, with no limits to the imagination regarding location, style, and additional objects, just like with image and video AIs:

Here is the corresponding prompt:

A vibrant 3D style, a delightful, fluffy creature leaping across a luminous rainbow bridge in a fantastical landscape. The creature is small and compact, with fur that mimics the warm hues of a sunrise—oranges, yellows, and pinks blending seamlessly together. Its most striking feature is a pair of large, upright ears, shaped like those of a German Shepherd, adding a playful contrast to its otherwise rounded form.

As it bounds across the rainbow on four short legs, its fur seems to flow and sway, giving it a sense of dynamism and energy. The rainbow bridge gracefully spans a whimsical landscape, perhaps with floating islands, glowing flora, and swirling clouds.

The lighting is bright and cheerful, bathing the creature and its surroundings in a warm glow. The overall impression is one of joy, wonder, and boundless energy—perfectly capturing the playful spirit of the creature and the magical nature of the world it inhabits. This image evokes a sense of childlike imagination, inviting the viewer to imagine the adventures that await this charming creature in its fantastical realm.

The worlds are also not statically generated, but dynamic events can also be scripted via prompt, such as changing weather, newly appearing objects, or characters. Further examples also show an interaction with the environment that leaves lasting traces:

Why?

Due to their interactivity, the worlds possess a completely different immersion than pure videos, which can now also be generated via video AI. The application possibilities are of course diverse, ranging from games that can be situated in any world desired by the user, to travels to other worlds or times, such as here to the Palace of Knossos:

And of course, completely new possibilities also arise for (AI) filmmaking, as a virtual world can be explored for the optimal shooting location and camera angle, or camera movements can be planned exactly.

To feel even more like the Holodeck, given the not-so-advanced holographic solutions, the journey via VR glasses is probably sufficiently immersive thanks to its 3D interactivity.

Many other use cases are also conceivable in the areas of learning, industry, and tourism, for example.

A Step on the Way to AGI

A particularly important application scenario is Genie 3 as a simulation environment for AI agents, who can gather experiences for the real world based on the interaction with the artificial world - much less complex than a complex real physical interaction via robot. The agent can recognize the visual world of Genie 3 and its objects and send commands to Genie 3 and thus interact and navigate in the world - and thus, for example, identify certain objects or also learn to navigate around obstacles or master difficult terrain. And they can be trained much faster in such artificial "real" environments than in reality (such as DeepMind&s universal game AI Alpha Zero, which was able to train itself through millions of runs in all possible games) - 100 times over and parallel in all possible concrete locations and tasks. An important step on the way to AGI, to Artificial General Intelligence, which possesses superhuman or at least human cognitive abilities in all intellectual task areas – not only in special, narrowly limited tasks like today&s AI systems.

What can Genie 3 (not yet) do?

DeepMind also provides information about the limitations that Genie 3 still has. For example, the scope of action of the agents is currently limited, as they can only perform a limited number of direct actions. Also, the realistic interaction and simulation of multiple independent agents in shared environments remains a challenge. In addition, Genie 3 cannot represent real geographic locations with complete accuracy. And a clear and readable text representation is usually only successful if the corresponding text is already contained in the input description. Finally, the duration of possible interactions is currently limited to a few minutes and does not yet allow for longer, continuous simulations.

But these are all limitations that will probably fall piece by piece in the next generations.

Bild zur Newsmeldung:

more infos at bei deepmind.google

deutsche Version dieser Seite: Google DeepMind Genie 3 - interaktive Welten in Echtzeit generieren