Monkey business: reinforcement learning meets neighborhood search for virtual network embedding.

Monkey Business : L'apprentissage par renforcement rencontre la recherche de quartier pour l'intégration de réseaux virtuels

Maxime Elkael (1, 2, 3) , Massinissa Ait Aba , Andrea Araldo (1, 4, 5) , Hind Castel-Taleb (1, 4, 5) , Badii Jouaber (1, 2, 3)

1 IP Paris - Institut Polytechnique de Paris
2 TSP - RS2M - Département Réseaux et Services Multimédia Mobiles
3 R3S-SAMOVAR - Réseaux, Systèmes, Services, Sécurité
4 TSP - RST - Département Réseaux et Services de Télécommunications
5 METHODES-SAMOVAR - Méthodes et modèles pour les réseaux

In this article, we consider the Virtual Network Embedding (VNE) problem for 5G networks slicing. This problem requires to allocate multiple Virtual Networks (VN) on a substrate virtualized physical network while maximizing among others, resource utilization, maximum number of placed VNs and network operator’s benefit. We solve the online version of the problem where slices arrive over time. Inspired by the Nested Rollout Policy Adaptation (NRPA) algorithm, a variant of the well known Monte Carlo Tree Search (MCTS) that learns how to perform good simulations over time, we propose a new algorithm that we call Neighborhood Enhanced Policy Adaptation (NEPA). The key feature of our algorithm is to observe NRPA cannot exploit knowledge acquired in one branch of the state tree for another one which starts differently. NEPA learns by combining NRPA with Neighborhood Search in a frugal manner which improves only promising solutions while keeping the running time low. We call this technique a monkey business because it comes down to jumping from one interesting branch to the other, similar to how monkeys jump from tree to tree instead of going down everytime. NEPA achieves better results in terms of acceptance ratio and revenue-to-cost ratio compared to other state-of-the-art algorithms, both on real and synthetic topologies.

Mots clés: Virtual network embedding; Reinforcement learning; Nested Rollout Policy Adaptation

Domaines: Informatique [cs]

Maxime Elkael, Massinissa Ait Aba, Andrea Araldo, Hind Castel-Taleb, Badii Jouaber. Monkey business: reinforcement learning meets neighborhood search for virtual network embedding.. Computer Networks, 2022, 216 (109204), ⟨10.1016/j.comnet.2022.109204⟩. ⟨hal-03876945⟩