The Multi-Task Learning View of Multimodal Data

Hachem Kadri (1) , Stéphane Ayache (1) , Cécile Capponi (1) , Sokol Koço (1) , François-Xavier Dupé (1) , Emilie Morvant (2, 1)

1 QARMA - éQuipe AppRentissage et MultimediA [Marseille]
2 LHC - Laboratoire Hubert Curien

We study the problem of learning from multiple views using kernel methods in a supervised setting. We approach this problem from a multi-task learning point of view and illustrate how to capture the interesting multimodal structure of the data using multi-task kernels. Our analysis shows that the multi-task perspective offers the flexibility to design more efficient multiple-source learning algorithms, and hence the ability to exploit multiple descriptions of the data. In particular, we formulate the multimodal learning framework using vector-valued reproducing kernel Hilbert spaces, and we derive specific multi-task kernels that can operate over multiple modalities. Finally, we analyze the vector-valued regularized least squares algorithm in this context, and demonstrate its potential in a series of experiments with a real-world multimodal data set.

Mots clés: multimodal learning; multiple views; multi-task kernels; vector-valued RKHS; cross-covariance operator

Domaines: Machine Learning [stat.ML]

Hachem Kadri, Stéphane Ayache, Cécile Capponi, Sokol Koço, François-Xavier Dupé, et al.. The Multi-Task Learning View of Multimodal Data. Asian Conference on Machine Learning (ACML), Nov 2013, Canberra, Australia. pp.261--276. ⟨hal-01070601⟩