|DynIBaR Can Freeze Time|
|Written by David Conrad|
|Sunday, 01 October 2023|
DynIBar aka Neural Dynamic Image-Based Rendering is a new approach to synthesizing novel views from mobile phone video footage. Not only does the technique eliminate blur and shake, it can even do bullet time effects to freeze time while sweeping the camera around to highlight a dramatic moment.
The paper “DynIBaR: Neural Dynamic Image-Based Rendering”, comes from Google Research and was awarded a best paper honorable mention at the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
To set the scene the researchers refer to recent advances in computer vision techniques to reconstruct and render static (non-moving) 3D scenes but point out that most of the videos people capture with their mobile devices depict moving objects, such as people, pets, and cars which lead to blurry inaccurate rendering when subject to standard view synthesis methods:
Referring to recent methods that use space-time neural radiance fields, such as Dynamic NeRFs developed at Cornell University by a team including some of the same researchers, we are told that such approaches still exhibit inherent limitations that prevent their application to casually captured, in-the-wild videos. In particular, they struggle to render high-quality novel views from videos featuring long time duration, uncontrolled camera paths and complex object motion. This is because of the need to store the entire moving scene in an MLP (MultiLayer Perceptron) data structure. The improvement achieved by DynIBaR is shown clearly here:
The video effects achieved by DyniBaR include:
All of these are demonstrated in this video:
DynIBaR: Neural Dynamic Image-Based Rendering
or email your comment to: firstname.lastname@example.org
|Last Updated ( Sunday, 01 October 2023 )|