Data Representation and Information in (Deep) Learning Networks: Special Issue Sensors

Abstract : The renewal of research interest in machine learning came with the emergence of the concept of big data during the late 2000s. Schematically, families of deep learning networks (DLN) emerged with industrial ambitions, taking advantage of the development of graphics cards (GPUs) to construct prediction models from massive amounts of collected and stored data and substantial means of calculation. It is illusory to want to learn a deep network involving millions of parameters without very large databases. We tend to think that more data lead to more information. In addition, the core of learning is all but a problem of data representation, not in the ‘data compression’ sense. For instance, in DLN, one representation (input layer) is replaced by a cascade of many representations (hidden layers), which means an increase of information (entropy). However, some questions remain: How does information spread in these inflationary networks? Is information transform conservative through the DLN? Can information theory quantify the learning capacity of these networks? This Special Issue has the ambition to collect responses to these questions.
Document type :
Directions of work or proceedings
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02398644
Contributor : Frédéric Davesne <>
Submitted on : Saturday, December 7, 2019 - 7:00:02 PM
Last modification on : Wednesday, January 8, 2020 - 1:34:02 AM

Identifiers

  • HAL Id : hal-02398644, version 1

Citation

Vincent Vigneron, Hichem Maaref. Data Representation and Information in (Deep) Learning Networks: Special Issue Sensors. 2020. ⟨hal-02398644⟩

Share

Metrics

Record views

46