Removing Reflections And Obstructions From Photos
Written by David Conrad   
Wednesday, 16 September 2015

Long gone are the days when you could say a photo never lies. The latest work by Google and MIT can produce perfect photos of things taken through glass windows and chain fences. It is another step on the way to pure computational photography where the final photo isn't something you could ever have actually seen.

We have previously reported on the automatic removal of reflections from photos using the slight displacement of the faint double image that results when you take a photo through glass in  Separating Reflection And Image. However, this new method is much more generally applicable. 

When humans look through a window or through an obstructing fence at a scene they often gain an impression of what things look like that aren't spoiled by the obscuring effects. The reason is that humans are able to subtract the "noise" by moving their head. The different viewpoints allow the distant scene to be viewed in its entirety and the brain does a reasonable job of "in-painting" the missing parts. 

This is all based on the different motion due to parallax that objects at different distances are subject to. When you move your head things that are close move more than things that are far away.

This is the principle that the Google and MIT teams use to separate the background scene from the foreground that is spoiling the imagery. 

photoobstruct1

The algorithm basically computes the optical flow of the imagery, i.e. how it moves, from a sequence of stills. The stills can be taken from a video or just taken as a set of photos as the camera is moved to say take a panorama. The principle may be simple, but the implementation is tour de force of image processing. 

The following video explains some of the details and shows lots of examples of how it all works: 

 

The removal of the fence is impressive, but so are the recovered images of what was reflected in the window. The technique could clearly be used not just to make better photos but to recover information that was previously hidden in photos. Given that so few frames are required to do the processing it could also be applied retrospectively.

We seem to be heading towards the day when the camera will be capable of taking photographs that simply couldn't have been taken with a non-computational approach. When you take out your camera and point it at something through a fence or a window and simply press the button to remove the imperfections, the result will be a photo that corresponds more closely to your experience of seeing but it will be a lie by any definition. 

photoobstruct2

Banner


Stanford Doggo - The Quadruped For The Rest Of Us
18/05/2019

Not everyone is going to be able to afford a Spot Mini when it becomes available, but a Stanford Doggo is much more within reach. The only snag is that "some assembly is required". See Doggo in action [ ... ]



Any Camera Can Capture Ultra Fast Action
12/05/2019

Ultra high speed videos of things like water droplets hitting the ground or balloons bursting are fascinating, but they need a high priced camera to render them as slowmo - or do they? Could computati [ ... ]


More News

 

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

 

 

Last Updated ( Wednesday, 16 September 2015 )