Datum-wise classification. A sequential Approach to sparsity

Gabriel Dulac-Arnold 1 Ludovic Denoyer 1 Philippe Preux 2, 3 Patrick Gallinari 1
1 MALIRE - Machine Learning and Information Retrieval
LIP6 - Laboratoire d'Informatique de Paris 6
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We propose a novel classification technique whose aim is to select an appropriate representation for each datapoint, in contrast to the usual approach of selecting a representation encompassing the whole dataset. This datum-wise representation is found by using a sparsity inducing empirical risk, which is a relaxation of the standard L 0 regularized risk. The classification problem is modeled as a sequential decision process that sequentially chooses, for each datapoint, which features to use before classifying. Datum-Wise Classification extends naturally to multi-class tasks, and we describe a specific case where our inference has equivalent complexity to a traditional linear classifier, while still using a variable number of features. We compare our classifier to classical L 1 regularized linear models (L 1-SVM and LARS) on a set of common binary and multi-class datasets and show that for an equal average number of features used we can get improved performance using our method.
Complete list of metadatas

Cited literature [15 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00772986
Contributor : Preux Philippe <>
Submitted on : Friday, January 11, 2013 - 2:19:53 PM
Last modification on : Thursday, March 21, 2019 - 1:07:49 PM
Long-term archiving on : Saturday, April 1, 2017 - 4:00:34 AM

File

ecml2011.pdf
Publisher files allowed on an open archive

Identifiers

Citation

Gabriel Dulac-Arnold, Ludovic Denoyer, Philippe Preux, Patrick Gallinari. Datum-wise classification. A sequential Approach to sparsity. ECML PKDD 2011 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2011, Athens, Greece. pp.375-390, ⟨10.1007/978-3-642-23780-5_34⟩. ⟨hal-00772986⟩

Share

Metrics

Record views

603

Files downloads

399