LTX Studio promises almost fully automated production of AI videos
[14:39 Tue,2.April 2024 by Thomas Richter]
We already reported on the virtual AI film studio LTX Studio around a month ago - now it has been officially unveiled. Lightricks has published even more information about LTX Studio than before, so that it is now even easier to imagine how the ambitious goal of offering a comprehensive solution for filmmaking using AI is to be realized.
Entire AI films should be able to be produced under one interface in the browser, from story development to (pre-)production and post-production. To this end, LTX Studio integrates AI tools for scripting, storyboarding, casting (i.e. consistent character generation), sound effects, voiceover, re-lighting, titling and automatic editing as well as, of course, video generation. LTX Studio promises two important features that are essential for AI films: Consistency and control over the objects displayed.
The workflow: AI total
The following demo clip is intended to show how quickly any video clip can be created with the help of LTX Studio. This should enable a completely new type of intuitive generation of films, which allows ideas to be very quickly converted into moving images including sound using only a text prompt and then interactively changed in order to adapt the resulting clips to your own ideas.
The future of digital storytelling is officially here.
Last night at our launch event, our VP Product presented this video, showcasing just how easy the creation process is with LTX Studio. If you have an idea, you can implement it into your story in just a few clicks. pic.twitter.com/kyyzsHNahq
Filmmaking with LTX Studio is highly automated - the workflow begins with a brief description of the story to be told, provided by the user, and the specification of the visual style. Based on this input, LTX Studio generates the script and storyboard. LTX Studio also suggests the characters, i.e. virtual actors (including appearance and voice), which can be changed by the user via a text prompt. More visual control is provided by the option of defining faces using your own photo.
In the next step, the storyboard created entirely by the software appears with the individual scenes of the film, which are further subdivided into the individual camera settings. The user can then change the individual camera settings and their sequence as a virtual director or cameraman.
LTX Studio user interface
Users can then manipulate individual aspects of each shot - the type of shot (such as Close-up, Medium Shots, Wide Shot, Establishing Shot, From Below, From Above, Bird's Eye View) as well as the scenery, lighting, weather, camera movement or movement of objects and any sound effects to accompany the scene (all via text prompt). Inpainting and magic erase functions are also available to modify individual shots for more precise control of the image content. As LTX Studio is not (yet) able to lip-sync, the generated voices of the actors are only available for a voice-over. If you don't like a shot or make a mistake that is too obvious, you simply "roll the dice" again and a new version of the shot is generated. Playing a movie in standard mode currently takes around 20 minutes - a higher resolution mode will also be available in the future.
Here is a more detailed clip from Theoretically Media that takes you step by step through the process of creating an AI movie with the help of LTX Studio:
Not yet ready for real AI movies
Lightricks combines an impressive arsenal of different AI tools under one roof in LTX Studio. Lightricks uses a mixture of open source models, licensed third-party services and its own technology for the various functions. However, the AI videos generated are more like an animated slideshow than a real video, with moving images lasting a maximum of 4 seconds, and also exhibit severe morphing artifacts. This shows the clear qualitative difference to the current state-of-the-art video AI, i.e. OpenAI Sora, both in terms of the length and consistency of the objects and their movements.
So what can you do with it? Due to its quality, the tool is currently recommended for storyboards, visual concepts or short surreal AI short films rather than photorealistic content. The previous demo clips from LTX Studio have more the character of somewhat strange movie trailers than real films. But - as the rapid development in the field of AI shows - this assessment could change in the coming months, let alone years. In any case, Lightricks is using LTX Studio to demonstrate what AI filmmaking could look like in the future once all the teething troubles have been ironed out.
The following demo clip - a mixture of real and AI-generated footage - by filmmaker Ari Folman demonstrates how LTX Studio can be used: for associative surreal dream sequences:
LTX Studio has now been "officially" launched, but the status has not yet changed - interested parties can still only gain access via a waitlist. The price is also not yet known, which will be decisive for future use and would have to be in line with Runway's pricing model in order to be competitive, where 450 seconds of Gen 2 AI video costs 28 dollars per month.
;){kind=logo}
;)