Neural Networks Learn To Remove Noise Without Seeing A Good Image
Written by Mike James   
Sunday, 15 July 2018

Don't you just hate film grain? Except, of course, when you don't and set it as a special effect on your digital camera. NVIDIA now has a tool that will remove grain, noise and all manner of defects, and the amazing part is that it learned how to without seeing a clean image. How?

From the point of view of machine learning this is almost magical. How can you teach a neural network to remove noise from an image when it is never given an example of what a clean image is. In other words, it isn't given the target it is aiming for. From the point of view of computational photography, or more generally signal processing, this is more than magic:

We observe that under suitable, common circumstances, we can learn to reconstruct signals from only corrupted examples, without ever observing clean signals,and often do this just as well as if we were using clean examples.

The idea is almost lunatic in its conception, but it has its roots going back to the use of the simple mean, or average, to estimate a value. The point is that, while the noise varies between images, the image remains constant. Thus, by training a network using nothing but noisy images, the network slowly learns to produce the average value of the images.

noisyimage1

To illustrate the idea the team took images from the well known ImageNet and added random noise to them. They then trained a neural network using just the corrupted images. That is, the network was attempting to reproduce a corrupted image using a different corrupt image as its input. Over time it learned to produce the average, or probably more accurately the invariant part, of the image. Then, when shown a corrupt image it hasn't been trained on, it produces a cleaned up version.

noisyimage2

This works much better than you could reasonably expect. See the video:

 

From simple Gaussian noise, the team went on to remove more complex types of corruption from the images. In particular, they had success removing a particularly difficult form of noise - Monte Carlo noise - that other methods have a tough time with.

"We have shown that simple statistical arguments lead to surprising new capabilities in learned signal recovery; it is possible to recover signals under complex corruptions without observing clean signals, at performance levels equal or close to using clean target data. There are several real-world situations where obtaining clean training data is difficult: low-light photography (e.g., astronomical imaging),physically-based image synthesis, and magnetic resonance imaging."

 

This is a powerful idea - you can learn the noise structure just from degraded samples. As long as you can get multiple samples of the noise, you don't need the original.

Is there something deeper going on here?

noisyimageicon

More Information

Noise2Noise: Learning Image Restoration without Clean Data

Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala and Timo Aila, NVIDIA, Aalto University, and MIT.

Related Articles

Using Super Seeing To See Photosynthesis

Selfie Righteous - Corrects Selfie Perspective

Computational Photography Shows Hi-Res Mars

Super Resolution

Seeing Buildings Shake With Software

See Invisible Motion, Hear Silent Sounds Cool? Creepy?

Extracting Audio By Watching A Potato Chip Packet

 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

Banner


Visual Studio Roadmap Promises Better Debugger
19/10/2018

Microsoft has updated its roadmaps for Visual Studio for Mac and Visual Studio 2019, promising a faster and more reliable code editor on the Mac, and a better debugger for VS 2019.



Explore More on Complexity Explorer
14/11/2018

There's a new facility on Complexity Explorer that lets you browse over 700 Complex Systems Science Resources. This will help anyone wanting to learn more locate relevant courses, tutorials, vide [ ... ]


More News

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 15 July 2018 )