GANalyze - What Makes Pictures Memorable?
Written by Sue Gee   
Sunday, 03 November 2019

While we can readily distinguish between more memorable and less memorable images, it's difficult to pin down exactly what affects this perception. Researchers at MIT have devised GANalyze to investigate what changes in “memorability" look like?

This senior co-authors of this research are Aude Oliva  and Phillip Isola whose earlier work on MemNet we reported back in 2015 in Are Your Pictures Memorable? Now they have extended their research into the visual features and properties of images that underlie high-level cognitive attributes like memorability, aesthetics, and emotional valence. 

Their new framework uses Generative Adversarial Networks (GANs) which we have repeatedly encountered in applications such as style transfer, repairing/restoring images and adding speech to avatars. In this instance GANs are applied to the problem of understanding high-level, cognitive image properties, such as memorability.

GANalyzemoremem

To obtain the results reported by their study MIT researchers Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola used the Generator of BigGAN which was pretrained on ImageNet. Its Assessor was MemNet, a CNN predicting image memorability. 

 

GANalyze

In addition to using the two popular automatic measures, the Frechet Inception Distance (FID) and the Inception Score (IS) the researchers conducted an experiment to collect human scores. They found that the images people remembered best had bright colors, simple backgrounds, and subjects centered prominently in the frame. They noted that a cheeseburger shifted to the far end of the memorability scale not only looks fatter and brighter but also “tastier,” while a panda stands out from its background and its black eyes, ears, and paws contrast sharply and with its white face.

GANalyzesq

Having verified that their method successfully discovers image manipulations that causally affect human memory performance, they went on to apply the same framework to analyze image aesthetics and emotional valence. to do this they reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable. 

If you want to know more about GANalze visit its website. If you want to experiement with it, it has a project repo on GitHub.

More Information

GANalyze: Toward Visual Definitions of Cognitive Image Properties Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola 

 

 

Related Articles

Are Your Pictures Memorable?

A Neural Net Colorizes Photos 

SparkleVision - Seeing Through The Glitter 

GANs Create Talking Avatars From One Photo

GANPaint: Using AI For Art

AI Makes Deep Fake News

Inventor Of GANs Joins Apple

The Flaw Lurking In Every Deep Neural Net 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.

Banner


Go Master Retires Citing AI Supremacy
01/12/2019

South Korean Go master Lee Sedol, the only human ever to score a win against AlphaGo, Google Deep Mind's AI Go player, has retired from the game on the grounds that, however well he plays, there  [ ... ]



AI for Oceans - Kids Use Computer Science For Good
09/12/2019

Code.org expands beyond teaching the concepts of coding. It moves on to artificial intelligence, AI. Computer Science Education Week starts today and with it Hour of Code events in schools around the  [ ... ]


More News

graphics

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 03 November 2019 )