Removing Reflections And Obstructions From Photos
Written by David Conrad   
Wednesday, 16 September 2015

Long gone are the days when you could say a photo never lies. The latest work by Google and MIT can produce perfect photos of things taken through glass windows and chain fences. It is another step on the way to pure computational photography where the final photo isn't something you could ever have actually seen.

We have previously reported on the automatic removal of reflections from photos using the slight displacement of the faint double image that results when you take a photo through glass in  Separating Reflection And Image. However, this new method is much more generally applicable. 

When humans look through a window or through an obstructing fence at a scene they often gain an impression of what things look like that aren't spoiled by the obscuring effects. The reason is that humans are able to subtract the "noise" by moving their head. The different viewpoints allow the distant scene to be viewed in its entirety and the brain does a reasonable job of "in-painting" the missing parts. 

This is all based on the different motion due to parallax that objects at different distances are subject to. When you move your head things that are close move more than things that are far away.

This is the principle that the Google and MIT teams use to separate the background scene from the foreground that is spoiling the imagery. 

photoobstruct1

The algorithm basically computes the optical flow of the imagery, i.e. how it moves, from a sequence of stills. The stills can be taken from a video or just taken as a set of photos as the camera is moved to say take a panorama. The principle may be simple, but the implementation is tour de force of image processing. 

The following video explains some of the details and shows lots of examples of how it all works: 

 

The removal of the fence is impressive, but so are the recovered images of what was reflected in the window. The technique could clearly be used not just to make better photos but to recover information that was previously hidden in photos. Given that so few frames are required to do the processing it could also be applied retrospectively.

We seem to be heading towards the day when the camera will be capable of taking photographs that simply couldn't have been taken with a non-computational approach. When you take out your camera and point it at something through a fence or a window and simply press the button to remove the imperfections, the result will be a photo that corresponds more closely to your experience of seeing but it will be a lie by any definition. 

photoobstruct2

Banner


Couchbase's Coding Assistant Goes GA
11/03/2024

Capella iQ, the AI coding assistant for developers that makes interacting with Couchbase using natural language possible, has gone from private beta to being generally available.



VLOGGER - AI Does Talking Heads
24/03/2024

Developed by Google researchers VLOGGER AI is a system that can create realistic videos of people talking and moving from a single still image and an audio clip as input. 


More News

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

 

 

Last Updated ( Wednesday, 16 September 2015 )