Towards a Publicly Available Framework to Process Traceroutes with MetaTrace

Matthieu Gouel (1, 2) , Maxime Mouchet , Omar Darwich (3) , Kevin Vermeulen (3)

1 LINCS - Laboratory of Information, Network and Communication Sciences
2 NPA - Networks and Performance Analysis
3 LAAS-SARA - Équipe Services et Architectures pour Réseaux Avancés

Various platforms, such as Ark, RIPE Atlas, and M-Lab, conduct millions of traceroutes daily and make them publicly accessible to researchers and operators. While these measurements are easily obtainable, the processing of such data poses challenges. Currently, there is no publicly available general framework designed specifically for storing and querying traceroutes. The conventional method of loading all traceroutes into memory becomes inadequate or necessitates costly hardware as the volume of traceroutes increases significantly. This research paper introduces an innovative utilization of the ClickHouse database, referred to as MetaTrace, and demonstrates its efficiency in querying and analyzing a dataset of 200 million traceroutes enriched with metadata within just 11 seconds, using a single server equipped with reasonably available resources. By making MetaTrace freely accessible as an open-source tool for the community, we aim to establish it as a significant stride toward standardizing traceroute processing. This standardization would enhance the replicability and reproducibility of research outcomes in this domain.

Domaines: Réseaux et télécommunications [cs.NI]

Matthieu Gouel, Maxime Mouchet, Omar Darwich, Kevin Vermeulen. Towards a Publicly Available Framework to Process Traceroutes with MetaTrace. 2023. ⟨hal-04198068⟩