Separating Reflection And Image
Written by David Conrad   
Monday, 25 May 2015

When you take a photo through glass the big problem is the reflections that you get from things inside the room. Unless you got to a lot of trouble to adjust the lighting these can ruin a shot. Now MIT researchers have a computational way of separating image from reflection. 

If you take a photo though glass then you will usually also capture reflections from inside the room or from outside if you are shooting inward. The problem of separating the two images, either because you want a clear view of the reflection or a perfect photo is a difficult problem. The image (I) that you get is a the sum of two components:

I= T + R

where T is the transmission and R the reflection. The problem is that you have no way of subtracting R because you don't know what R is. 

Previous solutions have involved either taking multiple photos or getting a human to indicate where the reflection is. However, a better solution is to notice that glass actually causes two reflection - one from the inside surface and one from the outside surface.

 

 

reflection1

 

This means that there are two copies of the reflected image shifted by a small amount. If the glass is thin and the camera far away then the image shift can be less than a pixel and of no use. If, however, the photo is taken through a double glazed unit then two reflections are easier to capture. 

What the new algorithm does it to take the image;

I = T + R + kR

where kR is the reflection shifted by k pixels. Now we have a chance of separating the components. 

The MIT researchers implemented an optimization process that can estimate k and R. The results are fairly impressive. Starting from a single image that has both a transmitted part and a reflected part it is possible extract the two components without out any human help:

reflection2

 

In many cases only the transmitted image is required:

reflection3

The recovered images tend not to be perfect, but they are better than the originals. The technique can be used to improve the quality of photographs where an object is reflected in a mirror, which also generates multiple ghost images. However, the most obvious use of the technique is the recovery of latent images in photos taken by security or crime-scene cameras. Discovering who took the photo, or any details of the room on the inside of the window that the photo was taken through, might well help an investigation. 

So remember the next time you take a photo through glass you could well be taking a selfie. 

reflection1

 

More Information

Reflection Removal using Ghosting Cues
YiChang Shih (MIT CSAIL), Dilip Krishnan (Google Research), Fredo Durand (MIT CSAIL) and William T. Freeman (MIT CSAIL)

Related Articles

Microsoft Hyperlapse Ready For Use

SparkleVision - Seeing Through The Glitter

Automatic High Dynamic Range (HDR) Photography

See Invisible Motion, Hear Silent Sounds Cool? Creepy?

Computational Camouflage Hides Things In Plain Sight

Google Has Software To Make Cameras Worse

Plenoptic Sensors For Computational Photography

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, FacebookGoogle+ or Linkedin,  or sign up for our weekly newsletter.

 

Banner


GitPod Adds Native GitLab Integration
15/10/2020

Users of GitLab can now make use of the Gitpod UI. GitLab is a popular DevOps tool that is used by more than 100,000 organizations. A partnership between GitLab and Gitpod means developers can make us [ ... ]



Eclipse IoT Developer Survey 2020
21/10/2020

The Eclipse Foundation’s IoT Working Group has released the results of its 2020 IoT Developer Survey. Agriculture emerges as the leading industry focus area and security as the top concern. Two thir [ ... ]


More News

square

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Monday, 25 May 2015 )