|GANalyze - What Makes Pictures Memorable?|
|Written by Sue Gee|
|Sunday, 03 November 2019|
While we can readily distinguish between more memorable and less memorable images, it's difficult to pin down exactly what affects this perception. Researchers at MIT have devised GANalyze to investigate what changes in “memorability" look like?
This senior co-authors of this research are Aude Oliva and Phillip Isola whose earlier work on MemNet we reported back in 2015 in Are Your Pictures Memorable? Now they have extended their research into the visual features and properties of images that underlie high-level cognitive attributes like memorability, aesthetics, and emotional valence.
Their new framework uses Generative Adversarial Networks (GANs) which we have repeatedly encountered in applications such as style transfer, repairing/restoring images and adding speech to avatars. In this instance GANs are applied to the problem of understanding high-level, cognitive image properties, such as memorability.
To obtain the results reported by their study MIT researchers Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola used the Generator of BigGAN which was pretrained on ImageNet. Its Assessor was MemNet, a CNN predicting image memorability.
In addition to using the two popular automatic measures, the Frechet Inception Distance (FID) and the Inception Score (IS) the researchers conducted an experiment to collect human scores. They found that the images people remembered best had bright colors, simple backgrounds, and subjects centered prominently in the frame. They noted that a cheeseburger shifted to the far end of the memorability scale not only looks fatter and brighter but also “tastier,” while a panda stands out from its background and its black eyes, ears, and paws contrast sharply and with its white face.
Having verified that their method successfully discovers image manipulations that causally affect human memory performance, they went on to apply the same framework to analyze image aesthetics and emotional valence. to do this they reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable.
GANalyze: Toward Visual Definitions of Cognitive Image Properties Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola
or email your comment to: firstname.lastname@example.org
|Last Updated ( Sunday, 03 November 2019 )|