|Megastereo - Panoramas With Depth|
|Written by David Conrad|
|Sunday, 23 June 2013|
Disney Research has made a breakthrough in implementing the technique of acquiring depth information from a simple camera scan of a scene. The images can also be stitched together to create hi-res panoramas with or without 3D information.
We all try our hand at a panorama shot - just pan the camera around and then use some autostitch software to put the mosaic together. Computational photography has some algorithms that will do the stitching automatically by automatically identifying the same features in each photo and then applying a transformation to take account of the changed point of view. For a perfect panorama you need to rotate the camera around its optical center, i.e. just rotate the camera.
However, if you just rotate the camera about itself you don't get any parallax effects - which is why it makes the stitching together easier. If you want to get 3D information from the sequence of shots you need parallax. This means rotating the camera mounted on an offset arm or just moving the camera along an arc in your outstretched hand. The big problem with this method is that the parallax makes it more difficult to fit the mosaic together, and this is the problem that the research team has been working on.
The good news is that the methods generalize to a range of camera positions and so to different types of panorama. To quote from the paper:
"As demonstrated in the results, our contributions resolve
Now it's time to watch the amazing video of what sorts of things their new technique can achieve:
The method is described in a paper to be presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 25-27, in Portland, Oregon. It is a complex multistep algorithm that has clearly taken a lot of work to get right. Essentially the extra work is attention to detail and using more sophisticated techniques to correct distortions and to work out the camera's position. The first set of steps corrects the images for lens and camera distortions. Next a flow based up-sampling is used to add parts of the image that might otherwise be missing in the panorama. This produces an artifact free stitching of the images together into a mono or stereo panorama.
"We demonstrated a solution for creating high-quality stereo panoramas at megapixel resolution. To this end, we made two main contributions: ﬁrst, we developed specifically tailored methods for correcting the input data. We proposed techniques to correct the camera orientations, remove undesired vertical parallax and to obtain a compact representation. Secondly, we use optical ﬂow to upsample the angular input resolution to generate the optimal number of rays for a given output resolution on the ﬂy, effectively resolving aliasing."
If you want to fine detail then read the paper. There are probably lots of real applications to which you could put this technique, but it still needs more computing power than you would find in a mobile phone.
A more general point is that it demonstrates just how much information there is in a set of images and how this can be used to create not just a panorama but a high resolution stereo panorama. Computational photography has only just got started.
Megastereo: Constructing High-Resolution Stereo Panoramas
To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin, or sign up for our weekly newsletter.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Sunday, 23 June 2013 )|