AI Used to Open Closed Eyes
Written by David Conrad   
Sunday, 24 June 2018

Facebook Researchers have used a novel approach for correcting photographs, specifically for in-painting open eyes when the subject of a photograph blinks or closes their eyes just as the shutter clicks.  

eyeinpaintingbanner

Photo editing software has been used for decades to correct red-eye in photographs and Adobe Photoshop now has an eye opening algorithm. To try to achieve more realistic results, Brian Dolhansky and Cristian Canton Ferrer used an artificial intelligence technique referred to as "Exemplar Generative Adversarial Networks" (ExGANs), that is:

a type of conditional GAN that utilize exemplar information to produce high-quality, personalized in-painting results. 

The extra, exemplar, information in this case is taken from another photo of the same subject.

The first figure in the paper reporting the research compares results from Adobe Photoshop and the ExGAN technique:

eyeinpaintingfif1

The second shows the general architecture of the Examplar GAN:

eyeinpaintinghow

The training flow is summarized as:

(1) mark the eyes from the input image

(2) in-paint the image with the reference image or code as a guide

(3) compute the gradient of the generator’s parameters via the content/ reconstruction loss between the input image and the in-painted image

(4) compute the gradient of the discriminator’s parameters with the in-painted image, another real, ground truth image, and the reference image or code

(5) backpropagate the discriminator error through the generator

(6) Optionally, the generator’s parameters can also be updated with a perceptual loss.

For reference-based Exemplar GANs, the compressing functions C(I) are the identity function.

In the concluding section of the paper the researchers write: 

Exemplar GANs provide a useful solution for image generation or in-painting, when a region of that image has some sort of identifying feature. They provide superior perceptual results because they incorporate identifying information stored in reference images or perceptual codes. A clear example of their capabilities is demonstrated by eye in-painting. Because Exemplar GANs are a general framework, they can be extended to other tasks within computer vision, and even to other domains.

 eyeinpaintingsq

More Information

Eye In-Painting with Exemplar Generative Adversarial Networks 

Related Articles

Facebook Advances in AI At F8

Facebook's New AI Lab In Montreal

Yann LeCun Recruited For Facebook's New AI Group 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.

 

 

Banner


A Raspberry Pi Hi Speed Camera For $6
10/08/2019

Taking high frame rate video is both fun and often useful, but a video camera that can do the job is very expensive. A small project involving a Raspberry Pi and its $6 camera has produced reasonably  [ ... ]



A Personal Sound Projector For $10
18/08/2019

Perhaps the "for $10" part is hype as that is just the cost of the webcam used in the tracking system, but it is low cost. Using acoustic meta-materials a team has managed to focus a beam of sound tha [ ... ]


More News

 

appC

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 24 June 2018 )