Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control

Ali El Amine (1, 2) , Paolo Dini (3) , Loutfi Nuaymi (1, 2)

1 IMT Atlantique - SRCD - Département Systèmes Réseaux, Cybersécurité et Droit du numérique
2 ADOPNET - Advanced technologies for operated networks
3 CTTC - Centre Tecnològic de Telecomunicacions de Catalunya = Telecommunications Technological Centre of Catalonia

In 5G networks, specific requirements are defined on the periodicity of Synchronization Signaling (SS) bursts. This imposes a constraint on the maximum period a Base Station (BS) can be deactivated. On the other hand, BS densification is expected in 5G architecture. This will cause a drastic increase in the network energy consumption followed by a complex interference management. In this paper, we study the Energy-Delay-Tradeoff (EDT) problem in a Heterogeneous Network (HetNet) where small cells can switch to different sleep mode levels to save energy while maintaining a good Quality of Service (QoS). We propose a distributed Q-learning algorithm controller for small cells that adapts the cell activity while taking into account the co-channel interference between the cells. Our numerical results show that multi-level sleep scheme outperforms binary sleep scheme with an energy saving up to 80% in the case when the users are delay tolerant, and while respecting the periodicity of the SS bursts in 5G.

Mots clés: Energy saving; delay; multi-level sleep mode; 5G; Q-learning

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]; Traitement du signal et de l'image [eess.SP]; Energie électrique

Ali El Amine, Paolo Dini, Loutfi Nuaymi. Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control. IEEE ICC 2020 Workshop on Cloud Technologies and Energy Efficiency in Mobile Communication Networks (CLEEN 2020) (IEEE ICC'20 Workshop - CLEEN), IEEE, Jun 2020, Dublin, Ireland. pp.6, ⟨10.1109/ICCWorkshops49005.2020.9145431⟩. ⟨hal-02615302⟩