Meta-data management of smart grids' data lake
Résumé
The context of this thesis is the specification of a general architecture of a data management system for smart grids.
The variety and large volumes of data, the diversity of systems and tools, and the velocity of treatments are the most important challenges in orchestrating a smart grid ecosystem. In addition, various analytical applications and business needs are emerging today at the crossing and the exploitation of data at several levels (raw and refined, current and historical, counters' measurement data and social data, etc.).
Metadata management thus makes it possible to exploit mechanisms of discovery and inference of data schemes in Smartgrid ecosystems in order to facilitate their integration / exploitation. To master the complexity of the proposed architectures for these ecosystems, the management of metadata is also central to avoid the appearance of silos causing loss of productivity, duplication of data and treatments, bad manipulations.
The research aims to:
** Provide a unified view of metadata: description of data sets (sizes, distribution of values, availability, location, diagrams, sizes, metrics, etc.), knowledge, links between data sets, ....
** Collect and aggregate metadata on data sets (in multiple formats and from multiple sources) and ecosystem services to facilitate integration and subsequent use
** Integrate a service dedicated to the automatic discovery and inference of schemas: schema evolution cycles, knowledge bases to keep track of inferred patterns and their matches, data indexing for multiple data sets Levels in the ecosystem, etc.
** Provide intelligent selection / retrieval of package and service information (based on construction path, uses, profiles, etc.).
The areas of application concern the optimization, self-adaptation, and governance of systems and data sets.