Deep learning for predicting image memorability

Hammad Squalli-Houssaini (1) , Ngoc Q. K. Duong (1) , Marquant Gwenaëlle (1) , Claire-Hélène Demarty (1)

1 Technicolor R & I [Cesson Sévigné]

Memorability of media content such as images and videos has recently become an important research subject in computer vision. This paper presents our computation model for predicting image memorability, which is based on a deep learning architecture designed for a classification task. We exploit the use of both convolutional neural network (CNN)-based visual features and semantic features related to image caption-ing for the task. We train and test our model on the large-scale benchmarking memorability dataset: LaMem. Experiment result shows that the proposed computational model obtains better prediction performance than the state of the art, and even outperforms human consistency. We further investigate the genericity of our model on other memorability datasets. Finally, by validating the model on interestingness datasets, we reconfirm the uncorrelation between memorability and in-terestingness of images. Index Terms— Image memorability, computational model, deep learning, interestingness, image captioning

Domains: Machine Learning [stat.ML]; Computer Vision and Pattern Recognition [cs.CV]

Hammad Squalli-Houssaini, Ngoc Q. K. Duong, Marquant Gwenaëlle, Claire-Hélène Demarty. Deep learning for predicting image memorability. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr 2018, Calgary, Canada. ⟨hal-01629297⟩