Spectral Norm Regularization of Orthonormal Representations for Graph Transduction

Rakesh Shivanna 1 Bibaswan Chatterjee 1 Raman Sankaran 1 Chiranjib Bhattacharyya 2 Francis Bach 3, 4
3 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : Recent literature [1] suggests that embedding a graph on an unit sphere leads to better generalization for graph transduction. However, the choice of optimal embedding and an efficient algorithm to compute the same remains open. In this paper, we show that orthonormal representations, a class of unit-sphere graph em-beddings are PAC learnable. Existing PAC-based analysis do not apply as the VC dimension of the function class is infinite. We propose an alternative PAC-based bound, which do not depend on the VC dimension of the underlying function class, but is related to the famous Lovász ϑ function. The main contribution of the paper is SPORE, a SPectral regularized ORthonormal Embedding for graph trans-duction, derived from the PAC bound. SPORE is posed as a non-smooth convex function over an elliptope. These problems are usually solved as semi-definite programs (SDPs) with time complexity O(n^6). We present, Infeasible Inexact prox-imal (IIP): an Inexact proximal method which performs subgradient procedure on an approximate projection, not necessarily feasible. IIP is more scalable than SDP, has an O(1 √ T) convergence, and is generally applicable whenever a suitable approximate projection is available. We use IIP to compute SPORE where the approximate projection step is computed by FISTA, an accelerated gradient descent procedure. We show that the method has a convergence rate of O(1 √ T). The proposed algorithm easily scales to 1000's of vertices, while the standard SDP computation does not scale beyond few hundred vertices. Furthermore, the analysis presented here easily extends to the multiple graph setting.
Type de document :
Communication dans un congrès
Neural Information Processing Systems, Dec 2015, Montreal, Canada
Liste complète des métadonnées

Littérature citée [21 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01265524
Contributeur : Francis Bach <>
Soumis le : lundi 1 février 2016 - 11:14:03
Dernière modification le : vendredi 17 août 2018 - 10:42:01
Document(s) archivé(s) le : vendredi 11 novembre 2016 - 23:53:42

Fichier

norm_graph_nips2015_.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01265524, version 1

Collections

Citation

Rakesh Shivanna, Bibaswan Chatterjee, Raman Sankaran, Chiranjib Bhattacharyya, Francis Bach. Spectral Norm Regularization of Orthonormal Representations for Graph Transduction. Neural Information Processing Systems, Dec 2015, Montreal, Canada. 〈hal-01265524〉

Partager

Métriques

Consultations de la notice

261

Téléchargements de fichiers

87