Vehicular Cloud Networking: Evolutionary Game with Reinforcement Learning based Access Approach
Résumé
Vehicular Ad hoc Networks (VANET) have recently known a growing interest due to their benefits for both drivers and passengers. In fact, safety and non-safety applications are provided to them which make their travels safer and comfortable. The emergence of many applications with different requirements needs better exploitation of the vehicular resources. In this context, vehicular cloud computing (VCC) is a new paradigm that aims to maximize the use of the vehicular capacities (storage, communication, and computation) opportunistically. It is based on the integration of VANET and cloud computing (CC): vehicles can share their resources or access the remote cloud to provide services. Both the access to the conventional cloud and the establishment of a temporary cloud present advantages and drawbacks: the important capacities of the CC present one of the advantages of access to the cloud while the cost of the cellular links is one of the drawbacks of access to the CC. In the case of a vehicular cloud (VC) composed of a set of vehicles, low cost and intermittent connections can be considered as the main benefit and concern, respectively. In this paper, we study the vehicular cloud access problem. We model it as an evolutionary game where the vehicles choose to cooperate or to access the conventional cloud through the LTE link. We focus on the centralized case, and we study the equilibrium of both homogeneous and heterogeneous players analytically. We propose an Evolutionary Game-based Vehicular Cloud Access algorithm (EG-VCA). In addition, we propose a distributed Q-learning based Vehicular Cloud Access algorithm (QL-VCA) that allows each vehicle to select the way of access independently to avoid the use of a centralized controller. The simulation results show that QL-VCA and EG-VCA algorithms present almost the same performances. In addition, they offer better results compared to the cases of using and accessing only the CC or the VC. Numerical results are also established. They outline the convergence of the two algorithms to the same state of equilibrium.