Many minutes of football matches have been televised in the last four weeks -- in 2D. You are not used to it any other way, and basically it works splendidly. But maybe the next World Cup will run on your living room table as a 3D reconstruction?
At the IEEE Conference on Computer Vision and Pattern Recognition, scientists from the University of Washington, Google and Facebook presented a system that converts a perfectly normal football recording, for example from YouTube, into a virtual, three-dimensional game display that can be viewed over a seemingly flat area in space via AR/Holo glasses. For this purpose, depth information is estimated for each player and their image is mixed with figures from football computer games in order to obtain realistic body poses. The calculations are done by a neural network.
As the demo material shows, the whole thing works reasonably well, but by no means without errors. You can now watch the game from different perspectives, but the figures disappear completely or partially for a moment and the ball is also usually not visible. But hey, you can´t have it all...
As an idea we actually find a 3D reconstruction not so bad, and considering that apparently no complex multicam environment was used for this system, the result would have to be significantly improved. It will be a while before the next World Cup -- maybe the required Holo/Mixed Reality technology will have prevailed by then.
