What makes an image memorable?
When glancing at a magazine or browsing the Internet we are continuously exposed to photographs and images. Despite this overflow of visual information, Aude Oliva’s group has shown that humans are extremely adept at remembering thousands of pictures and a surprising amount of their visual details (PNAS, 2008; JEP:G, 2010; Psychological Science, 2010). But whereas some images stick in our minds, others are ignored or quickly forgotten. Artists, advertisers and photographers are routinely challenged by the question “what makes a picture memorable?” and are then presented with the task of how to create an image that speaks to the viewer. The quality of what makes an image memorable is hard to quantify, yet our work suggests that it is not an inexplicable phenomenon, and that subjective intuitions of what make an image memorable may need to be revised (next article to appear in 2014 in IEEE Pattern Analysis and Machine Intelligence - PAMI)
For instance, look at the photographs above. Which images do you think are more memorable? (see answers in footnote). The memorability of these images is consistent across subjects, making some images intrinsically memorable, independent of the subjects' past experiences or biases. By analyzing a variety of image features and objects that might contribute to visual memorability, we found some intuitive results -- pictures with people and central objects in them are highly memorable -- and some surprising results -- beautiful landscapes are among the least memorable (see our CVPR 2011 paper). Based on our findings we also developed an algorithm that can take a new image, and automatically predict how likely it is to be remembered. Our image ranking algorithm matches empirical data quite nicely: it picks out dynamic scenes with people interacting as most memorable, static indoor environments and human-scale objects as somewhat less memorable, and outdoor vistas as forgettable.
In a follow up paper at NIPS 2011, we augment the object and scene level annotations with interpretable attributes that describe various spatial, content and aesthetic properties of images. We find a compact set of features that characterize the memorability of any individual image: for instance, images of enclosed spaces containing people with visible-faces are memorable, while images of vistas and peaceful looking places are not. Contrary to popular belief, unusual or aesthetically pleasing scenes are not highly memorable ! This line of work represents the first attempts at determining intrinsic image memorability, and opens a new domain of investigation at the interface between human cognition and computer vision.
Most recently, the team has investigated what are the features that make a photograph of a face memorable, and two articles are in press (Journal of Experimental Psychology: General, and ICCV 2013). Every day, we encounter an overwhelming number of photographs and images of people's faces. Many inter-personal interactions are mediated by such images: we view people's Facebook profile pictures; memorize photographs of our students; browse personals on dating websites; skim through pictures attached to job applications; and encounter countless face images published on advertisements on billboards, in magazines, and online. Furthermore, the vast majority of these face images are intended to be remembered, either because of personal relevance, because the processing of information would become more efficient, or because the images were deliberately designed to be memorable.
Future development of such automatic algorithms could have many exciting and far-reaching applications, particularly in domains related to education, learning and training. For example, which graph or schema for a textbook or a lecture would be memorable? In a recent article, the team looked at what makes a visualization memorable. Out of a multitude of photos taken on a family vacation, which one will be most memorable? Which photo should one choose to represent their business on their Facebook page? What logo created by a graphic designer will be remembered by more people? What photograph for a book cover will sell more copies? An automatic algorithm would be able to browse an infinite number of images and videos, effectively guiding the choices of the user, helping the learner, or it could continuously report on the memorability of visual events in our hectic, 24- hour news cycle world, weeding out less-effective images and instead focusing on this with the most memorable impacts. And one more thing: how many “highly” memorable images can the brain store? According to our study in Journal of Experimental Psychology: General, 2010 (Konkle et al., 2010), the answer is “it could be limitless, as long as each image has a unique “conceptual” code”.
In addition to applications in the gaming and entertainment industries (e.g for building visual memory games and more efficient mnemonic techniques), the impact of such algorithms in education, clinical research and re-education could be phenomenal: measuring visual memory degradation, and understanding more precisely what aspects of visual memory are deficient in specific psychological or brain disorders is an expanding area of research. This could also be applied to uncovering answers to some of the most basic questions of cognitive science and neuroscience: how can we recall such an amazing density of visual information and in what ways is this recall still fallible? What are the neural structures subtending the massive visual memory capacity, and how can we preserve or enhance (for patients suffering from Alzheimer) this capacity?
- Oliva, A., Isola, P., Khosla, A., & Bainbridge, W.A. (2013). What makes a picture memorable?. SPIE Newsroom article, 7 May 2013, DOI: 10.1117/2.1201304.004806 article SPIE website
- Borkin, M.A, Vo, A.A., Bylinskii, Z., Isola, P., Sunkavalli, S., Oliva, A., & Pfister, H. (2013). What makes a visualization memorable? .IEEE Transactions on Visualization and Computer Graphics. abstract article
- Bainbridge, W. A., Isola, P., & Oliva, A. (in press). The Intrinsic Memorability of Face Images. Journal of Experimental Psychology: General. abstract
- Khosla, A., Bainbridge, W. A., Torralba, A., & Oliva, A. (in press). Modifying the Memorability of Face Photographs. Proceedings of the International Conference on Computer Vision (ICCV) abstract
- Isola, P., Xiao, J., Parikh, D, Torralba, A., & Oliva, A. (in press). What makes a photograph memorable?. IEEE Pattern Analysis and Machine Intelligence (PAMI). abstract article
- Brady, T.F., Konkle, T., Alvarez, G.A, & Oliva, A. (2013). Real-world objects are not represented as bound units: Independent forgetting of different object details from visual memory. Journal of Experimental Psychology: General, 142(3), 791-808. abstract article stimuli
- Brady, T.F., Konkle, T., Gill, J., Oliva, A., & Alvarez, G.A. (2013). Long-term memory has the same limit on fidelity as visual working memory. Psychological Science, 24(6), 981-990abstract article website
- Khosla, A., Xiao, J., Torralba, A., & Oliva, A. (2012). Memorability of Image Regions.Advances in Neural Information Processing Systems, P. Bartlett and F.C.N. Pereira and C.J.C. Burges and L. Bottou and K.Q. Weinberge (Eds.), 25 ,305-313 article posterwebsite
- Isola, P., Xiao, J., Torralba, A., & Oliva, A. (2011). What makes an image memorable? Proceedings of the 24rd IEEE Conference on Computer Vision and Pattern Recognition (pp. 145-152). abstract article poster website
- Isola, P., Parikh, D., Torralba, A., & Oliva, A. (2011). Understanding the Intrinsic Memorability of Images. Advances in Neural Information Processing Systems, J. Shawe-Taylor and R.S. Zemel and P. Bartlett and F.C.N. Pereira and K.Q. Weinberger (Eds.), 24, 2429-2437 abstract article poster website
- Konkle, T.*, Brady, T.F.*, Alvarez, G.A., & Oliva, A. (2010). Scene memory is more detailed than you think: the role of categories in visual long-term memory. Psychological Science, 21(11), 1551-1556. abstractarticlewebsite
- Konkle, T., Brady, T.F., Alvarez, G.A & Oliva, A. (2010). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139 (3), 558-578. abstractwebsitearticle
- Brady, T.F., Konkle, T., Alvarez, G.A., & Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object details. Proceedings of the National Academy of Sciences, USA, vol 105 (38), 14325-14329.abstractwebsitearticle
Footnote: Images (a,d,e) are among the most memorable images, while (b,c,f) are among the least memorable