Megastereo - Panoramas With Depth
Written by David Conrad   
Sunday, 23 June 2013

Disney Research has made a breakthrough in implementing the technique of acquiring depth information from a simple camera scan of a scene. The images can also be stitched together to create hi-res panoramas with or without 3D information.




We all try our hand at a panorama shot - just pan the camera around and then use some autostitch software to put the mosaic together. Computational photography has some algorithms that will do the stitching automatically by automatically identifying the same features in each photo and then applying a transformation to take account of the changed point of view. For a perfect panorama you need to rotate the camera around its optical center, i.e. just rotate the camera. 

However, if you just rotate the camera about itself you don't get any parallax effects - which is why it makes the stitching together easier. If you want to get 3D information from the sequence of shots you need parallax. This means rotating the camera mounted on an offset arm or just moving the camera along an arc in your outstretched hand. The big problem with this method is that the parallax makes it more difficult to fit the mosaic together, and this is the problem that the research team has been working on. 




The good news is that the methods generalize to a range of camera positions and so to different types of panorama. To quote from the paper:

"As demonstrated in the results, our contributions resolve
central issues of existing techniques for both stereo- and
monoscopic panorama generation, as well as for any multiperspective imaging method based on stitching, like x-slit, pushbroom or general linear cameras."

Now it's time to watch the amazing video of what sorts of things their new technique can achieve:



The method is described in a paper to be presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 25-27, in Portland, Oregon. It is a complex multistep algorithm that has clearly taken a lot of work to get right. Essentially the extra work is attention to detail and using more sophisticated techniques to correct distortions and to work out the camera's position. The first set of steps corrects the images for lens and camera distortions. Next a flow based up-sampling is used to add parts of the image that might otherwise be missing in the panorama. This produces an artifact free stitching of the images together into a mono or stereo panorama. 

"We demonstrated a solution for creating high-quality stereo panoramas at megapixel resolution. To this end, we made two main contributions: first, we developed specifically tailored methods for correcting the input data. We proposed techniques to correct the camera orientations, remove undesired vertical parallax and to obtain a compact representation. Secondly, we use optical flow to upsample the angular input resolution to generate the optimal number of rays for a given output resolution on the fly, effectively resolving aliasing."

If you want to fine detail then read the paper. There are probably lots of real applications to which you could put this technique, but it still needs more computing power than you would find in a mobile phone. 




A more general point is that it demonstrates just how much information there is in a set of images and how this can be used to create not just a panorama but a high resolution stereo panorama. Computational photography has only just got started.


More Information

Megastereo: Constructing High-Resolution Stereo Panoramas 
Christian Richardt (Disney Research, Zurich; REVES/INRIA Sophia Antipolis), Yael Pritch (Disney Research, Zurich), Henning Zimmer (Disney Research, Zurich; ETH Zurich), Alexander Sorkine-Hornung (Disney Research, Zurich)

Related Articles

Imaging The Pulse

Google - We Have Ways Of Making You Smile

Computational Photography On A Chip

Super Seeing Software Ready To Download

Blink If You Don't Want To Miss it


To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, FacebookGoogle+ or Linkedin,  or sign up for our weekly newsletter.


kotlin book



or email your comment to:



Look Once to Hear - A Spy's Dream Come True

Deep learning has triumphed again. You can don a pair of headphones, look at a person talking and from then on the system will track the person so you can hear them as they move away or become swamped [ ... ]

Let Oracle's Coding Assistant Do The Grunt Work

Oracle is getting into the coding assistants game. The aptly named "Oracle Code Assist" is going to be optimized for Java, SQL, and Oracle Cloud-based applications.

More News

Last Updated ( Sunday, 23 June 2013 )