Removing Reflections And Obstructions From Photos
Removing Reflections And Obstructions From Photos
Written by David Conrad   
Wednesday, 16 September 2015

Long gone are the days when you could say a photo never lies. The latest work by Google and MIT can produce perfect photos of things taken through glass windows and chain fences. It is another step on the way to pure computational photography where the final photo isn't something you could ever have actually seen.

We have previously reported on the automatic removal of reflections from photos using the slight displacement of the faint double image that results when you take a photo through glass in  Separating Reflection And Image. However, this new method is much more generally applicable. 

When humans look through a window or through an obstructing fence at a scene they often gain an impression of what things look like that aren't spoiled by the obscuring effects. The reason is that humans are able to subtract the "noise" by moving their head. The different viewpoints allow the distant scene to be viewed in its entirety and the brain does a reasonable job of "in-painting" the missing parts. 

This is all based on the different motion due to parallax that objects at different distances are subject to. When you move your head things that are close move more than things that are far away.

This is the principle that the Google and MIT teams use to separate the background scene from the foreground that is spoiling the imagery. 

photoobstruct1

The algorithm basically computes the optical flow of the imagery, i.e. how it moves, from a sequence of stills. The stills can be taken from a video or just taken as a set of photos as the camera is moved to say take a panorama. The principle may be simple, but the implementation is tour de force of image processing. 

The following video explains some of the details and shows lots of examples of how it all works: 

 

The removal of the fence is impressive, but so are the recovered images of what was reflected in the window. The technique could clearly be used not just to make better photos but to recover information that was previously hidden in photos. Given that so few frames are required to do the processing it could also be applied retrospectively.

We seem to be heading towards the day when the camera will be capable of taking photographs that simply couldn't have been taken with a non-computational approach. When you take out your camera and point it at something through a fence or a window and simply press the button to remove the imperfections, the result will be a photo that corresponds more closely to your experience of seeing but it will be a lie by any definition. 

photoobstruct2

Banner


Microsoft Kills The Kinect
29/10/2017

This is a very strange "end of era" story. The Microsoft Kinect 3D sensor is dead and gone. How can such a ground breaking device be removed? Before you ask, there doesn't seem to be an alternative.&n [ ... ]



AWS Amplify Opens Cloud Services For JavaScript
23/11/2017

Amazon has released an open source library for interacting with cloud services that use JavaScript applications. The AWS Amplify library is organized into a number of categories, with more planne [ ... ]


More News

 

 
 

 

blog comments powered by Disqus

 

 

Last Updated ( Wednesday, 16 September 2015 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2017 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.