ToonCrafter is an AI project on GitHub that aims to solve the problem of interpolating animated videos in a new way. The aim is to generate several images between two drawn frames in order to create a fluid movement for the eye between these two images. Normal interpolation between the frames does not work here, as this would look like warping at best. And previously used algorithms often have problems with the exaggerated non-linear and large movements of typical animations as well as the overlapping of objects (occlusion) - which ultimately almost always leads to implausible and/or completely failed interpolation results.
ToonCrafter - Generative AI interframe generation for animated videos
Generative AI methods, on the other hand, have already proven several times in recent months that they can theoretically predict the change and movement of objects much better with their advanced understanding of image content. And could therefore be well suited for interpolation in this problem area.
ToonCrafter now offers just such a generative cartoon film interpolation. The model was specially trained for animated films and now seems to deliver quite impressive results. However, critical users on Reddit note that the tool is currently limited to 16 frames per second at a resolution of 320x512 and that the examples presented are only ‘rare sultanas’. In most real-life cases, the output is still unusable. But these probably very carefully selected raisins look remarkably good:
ToonCrafter basically demonstrates the amazing possibilities of generative AI models for intermediate image generation - and that these models will continue to improve significantly. So if development does not stop here and now, it is foreseeable that in the future it will be possible to generate animated cartoons from just a few still images.
And that's not where generative AI ends, of course. Because even with real still images, high-quality interframe animation is not only conceivable, but will probably become a reality in the coming months or years. And anyone who has followed recent developments will know that months is the more likely forecast horizon here.
