M. Caron, P. Bojanowski, J. Mairal, and A. Joulin, Unsupervised pre-training of image features on non-curated data, Proceedings of the International Conference on Computer Vision (ICCV), 2019.
URL : https://hal.archives-ouvertes.fr/hal-02119564

P. Goyal, D. Mahajan, A. Gupta, and I. Misra, Scaling and benchmarking self-supervised visual representation learning, Proceedings of the International Conference on Computer Vision (ICCV), 2019.

P. Bachman, R. D. Hjelm, and W. Buchwalter, Learning representations by maximizing mutual information across views, 2019.

O. J. Hénaff, A. Razavi, C. Doersch, S. Eslami, and A. V. Oord, Data-efficient image recognition with contrastive predictive coding, 2019.

K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

Y. Lecun, J. S. Denker, and S. A. Solla, Optimal brain damage, Advances in Neural Information Processing Systems (NIPS), 1990.

C. Louizos, M. Welling, and D. P. Kingma, Learning sparse neural networks through l 0 regularization, International Conference on Learning Representations (ICLR), 2018.

S. Han, J. Pool, J. Tran, and W. Dally, Learning both weights and connections for efficient neural network, Advances in Neural Information Processing Systems (NIPS), 2015.

M. Noroozi and P. Favaro, Unsupervised learning of visual representations by solving jigsaw puzzles, Proceedings of the European Conference on Computer Vision (ECCV), 2016.

S. Gidaris, P. Singh, and N. Komodakis, Unsupervised representation learning by predicting image rotations, International Conference on Learning Representations (ICLR), 2018.
URL : https://hal.archives-ouvertes.fr/hal-01864755

J. Frankle and M. Carbin, The lottery ticket hypothesis: Finding sparse, trainable neural networks, International Conference on Learning Representations (ICLR), 2019.

J. Frankle, G. K. Dziugaite, D. M. Roy, and M. Carbin, Linear mode connectivity and the lottery ticket hypothesis, 2019.

Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell, Rethinking the value of network pruning, International Conference on Learning Representations (ICLR), 2019.

A. S. Morcos, H. Yu, M. Paganini, and Y. Tian, One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers, Advances in Neural Information Processing Systems (NeurIPS), 2019.

H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, Pruning filters for efficient convnets, International Conference on Learning Representations (ICLR, 2017.

A. Prakash, J. Storer, D. Florencio, and C. Zhang, Repr: Improved training of convolutional filters, Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Y. Guo, A. Yao, and Y. Chen, Dynamic network surgery for efficient dnns, Advances in Neural Information Processing Systems (NIPS), 2016.

X. Dong, S. Chen, and S. Pan, Learning to prune deep neural networks via layer-wise optimal brain surgeon, Advances in Neural Information Processing Systems (NIPS), 2017.

K. Ullrich, E. Meeds, and M. Welling, Soft weight-sharing for neural network compression, International Conference on Learning Representations, 2017.

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, Xnornet: Imagenet classification using binary convolutional neural networks, Proceedings of the European Conference on Computer Vision (ECCV), 2016.

W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen, Compressing neural networks with the hashing trick, Proceedings of the International Conference on Machine Learning (ICML), 2015.

H. Zhou, J. Lan, R. Liu, and J. Yosinski, Deconstructing lottery tickets: Zeros, signs, and the supermask, Workshop on Identifying and Understanding Deep Learning Phenomena (ICML), 2019.

H. Yu, S. Edunov, Y. Tian, and A. S. Morcos, Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP, International Conference on Learning Representations (ICLR), p.2020

C. Doersch, A. Gupta, and A. A. Efros, Unsupervised visual representation learning by context prediction, Proceedings of the International Conference on Computer Vision (ICCV), 2015.

X. Wang and A. Gupta, Unsupervised learning of visual representations using videos, Proceedings of the International Conference on Computer Vision (ICCV), 2015.

R. Zhang, P. Isola, and A. A. Efros, Colorful image colorization, Proceedings of the European Conference on Computer Vision (ECCV), 2016.

D. Pathak, R. Girshick, P. Dollár, T. Darrell, and B. Hariharan, Learning features by watching objects move, Proceedings of the Conference on Computer Vision and Pattern Recognition, 2017.

Y. Tian, D. Krishnan, and P. Isola, Contrastive multiview coding, 2019.

A. Dosovitskiy, P. Fischer, J. T. Springenberg, M. Riedmiller, and T. Brox, Discriminative unsupervised feature learning with exemplar convolutional neural networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016.

P. Bojanowski and A. Joulin, Unsupervised learning by predicting noise, Proceedings of the International Conference on Machine Learning (ICML), 2017.

Z. Wu, Y. Xiong, S. X. Yu, and D. Lin, Unsupervised feature learning via non-parametric instance discrimination, Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

M. Caron, P. Bojanowski, A. Joulin, and M. Douze, Deep clustering for unsupervised learning of visual features, Proceedings of the European Conference on Computer Vision (ECCV), 2018.

C. Doersch and A. Zisserman, Multi-task self-supervised visual learning, Proceedings of the International Conference on Computer Vision (ICCV, 2017.

J. Deng, W. Dong, R. Socher, L. Li, K. Li et al., Imagenet: A large-scale hierarchical image database, Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

A. Krizhevsky, Learning multiple layers of features from tiny images, 2009.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems (NIPS), 2012.

K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014.

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang et al., Automatic differentiation in pytorch, 2017.

M. Everingham, L. Van-gool, C. K. Williams, J. Winn, and A. Zisserman, The pascal visual object classes (voc) challenge, International Journal of Computer Vision (IJCV), 2010.

B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva, Learning deep features for scene recognition using places database, Advances in Neural Information Processing Systems (NIPS), 2014.

L. Jing and Y. Tian, Self-supervised visual feature learning with deep neural networks: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.

X. Zhai, A. Oliver, A. Kolesnikov, and L. Beyer, S4l: Selfsupervised semi-supervised learning, Proceedings of the International Conference on Computer Vision (ICCV), 2019.

T. Miyato, S. Maeda, M. Koyama, and S. Ishii, Virtual adversarial training: a regularization method for supervised and semisupervised learning, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.

D. Lee, Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks, Workshop on Challenges in Representation Learning, (ICML), 2013.

I. Loshchilov and F. Hutter, Sgdr: Stochastic gradient descent with warm restarts, 2016.