GANalyze - What Makes Pictures Memorable?
Written by Sue Gee   
Sunday, 03 November 2019

While we can readily distinguish between more memorable and less memorable images, it's difficult to pin down exactly what affects this perception. Researchers at MIT have devised GANalyze to investigate what changes in “memorability" look like?

This senior co-authors of this research are Aude Oliva  and Phillip Isola whose earlier work on MemNet we reported back in 2015 in Are Your Pictures Memorable? Now they have extended their research into the visual features and properties of images that underlie high-level cognitive attributes like memorability, aesthetics, and emotional valence. 

Their new framework uses Generative Adversarial Networks (GANs) which we have repeatedly encountered in applications such as style transfer, repairing/restoring images and adding speech to avatars. In this instance GANs are applied to the problem of understanding high-level, cognitive image properties, such as memorability.

GANalyzemoremem

To obtain the results reported by their study MIT researchers Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola used the Generator of BigGAN which was pretrained on ImageNet. Its Assessor was MemNet, a CNN predicting image memorability. 

 

GANalyze

In addition to using the two popular automatic measures, the Frechet Inception Distance (FID) and the Inception Score (IS) the researchers conducted an experiment to collect human scores. They found that the images people remembered best had bright colors, simple backgrounds, and subjects centered prominently in the frame. They noted that a cheeseburger shifted to the far end of the memorability scale not only looks fatter and brighter but also “tastier,” while a panda stands out from its background and its black eyes, ears, and paws contrast sharply and with its white face.

GANalyzesq

Having verified that their method successfully discovers image manipulations that causally affect human memory performance, they went on to apply the same framework to analyze image aesthetics and emotional valence. to do this they reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable. 

If you want to know more about GANalze visit its website. If you want to experiement with it, it has a project repo on GitHub.

More Information

GANalyze: Toward Visual Definitions of Cognitive Image Properties Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola 

 

 

Related Articles

Are Your Pictures Memorable?

A Neural Net Colorizes Photos 

SparkleVision - Seeing Through The Glitter 

GANs Create Talking Avatars From One Photo

GANPaint: Using AI For Art

AI Makes Deep Fake News

Inventor Of GANs Joins Apple

The Flaw Lurking In Every Deep Neural Net 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Android Studio Iguana With Crash Reports
05/03/2024

Google has announced that the latest version of Android Studio, Iguana, is now stable. It has version control system support in App Quality Insights and new built-in support for creating baseline prof [ ... ]



Microsoft Is Ending Support For Windows Android Subsystem
07/03/2024

Microsoft has announced that it is ending support for running Android applications under Windows 11. The support will cease next year. This announcement was followed rapidly by Amazon announcing the e [ ... ]


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 03 November 2019 )