To make human movements look real in animations, they are digitized via motion capturing and can thus be reproduced with the figure to be animated Model? However, a problem is not only the (physically, anatomically and contextually) meaningful combination of motion sequences or even individual elements such as the movements of the upper and lower body, but also the grounding - the feet often appear to float.
But here, too, the DeepLearning approach is extremely promising: instead of defining rules for all possible cases as to which movement may follow which, an AI algorithm learns to estimate the correct sequences based on training material from movements of real people.
Sebastian Starke&s new algorithm uses 20 hours of motion capture training data to "understand" human martial arts movements and synthesize not only natural-looking sequences of movements but also new "moves." For example, the AI learns from training footage of a boxing attack and a defensive stance and then combines them into a move that incorporates both, if desired. Also realistically simulated are interactions with an opponent such as reactions to hits, dodging an opponent&s attack, or a joint clinch.
And the whole thing doesn&t remain abstract: the movements can easily be controlled with a game controller and displayed in real time - ideal for real-time applications like computer games. And the beauty is that the DeepLearning algorithm enables universality of the simulated movements. Thus, simply by using different training material instead of martial arts movements, completely different types of natural human movement patterns can be learned and synthesized, such as dance movements, which are also often performed in interaction, but with a partner instead of an opponent.
And here the algorithm is explained again nicely by Károly Zsolnai-Fehér of 2 Min Papers: