Skip to Main content Skip to Navigation
Conference papers

Bootstrap Your Own Latent: A new approach to self-supervised learning

Abstract : We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.
Document type :
Conference papers
Complete list of metadata

Cited literature [66 references]  Display  Hide  Download

https://hal.inria.fr/hal-02869787
Contributor : Michal Valko <>
Submitted on : Tuesday, June 16, 2020 - 11:53:59 AM
Last modification on : Friday, February 26, 2021 - 10:36:30 AM

File

grill2020bootstrap.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02869787, version 1
  • ARXIV : 1910.10945

Citation

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, et al.. Bootstrap Your Own Latent: A new approach to self-supervised learning. Neural Information Processing Systems, 2020, Montréal, Canada. ⟨hal-02869787v1⟩

Share

Metrics

Record views

278

Files downloads

712