Exploiting Negative Evidence for Deep Latent Structured Models

Abstract : The abundance of image-level labels and the lack of large scale detailed annotations (e.g. bounding boxes, segmentation masks) promotes the development of weakly supervised learning (WSL) models. In this work, we propose a novel framework for WSL of deep convolutional neural networks dedicated to learn localized features from global image-level annotations. The core of the approach is a new latent structured output model equipped with a pooling function which explicitly models negative evidence, e.g. a cow detector should strongly penalize the prediction of the bedroom class. We show that our model can be trained end-to-end for different visual recognition tasks: multi-class and multi-label classification, and also structured average precision (AP) ranking. Extensive experiments highlight the relevance of the proposed method: our model outperforms state-of-the art results on six datasets. We also show that our framework can be used to improve the performance of state-of-the-art deep models for large scale image classification on ImageNet. Finally, we evaluate our model for weakly supervised tasks: in particular, a direct adaptation for weakly supervised segmentation provides a very competitive model.
Document type :
Journal articles
Complete list of metadatas

Cited literature [80 references]  Display  Hide  Download

https://hal.sorbonne-universite.fr/hal-01969819
Contributor : Gestionnaire Hal-Upmc <>
Submitted on : Friday, January 4, 2019 - 3:22:36 PM
Last modification on : Friday, July 5, 2019 - 3:26:03 PM
Long-term archiving on : Friday, April 5, 2019 - 2:46:13 PM

File

2018PAMIcord_sans marque.pdf
Files produced by the author(s)

Identifiers

Citation

Thibaut Durand, Nicolas Thome, Matthieu Cord. Exploiting Negative Evidence for Deep Latent Structured Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2019, 41 (2), pp.337-351. ⟨10.1109/TPAMI.2017.2788435⟩. ⟨hal-01969819⟩

Share

Metrics

Record views

33

Files downloads

86