Reinforcement learning-based cell selection in sparse mobile crowdsensing

Abstract : Sparse Mobile Crowdsensing (MCS) is a novel MCS paradigm which allows us to use the mobile devices to collect sensing data from only a small subset of cells (sub-areas) in the target sensing area while intelligently inferring the data of other cells with quality guarantee. Since selecting sensed data from different cell sets will probably lead to diverse levels of inference data quality, cell selection (i.e., choosing which cells in the target area to collect sensed data from participants) is a critical issue that will impact the total amount of data that requires to be collected (i.e., data collection costs) for ensuring a certain level of data quality. To address this issue, this paper proposes the reinforcement learning-based cell selection algorithm for Sparse MCS. First, we model the key concepts in reinforcement learning including state, action, and reward, and then propose a Q-learning based cell selection algorithm. To deal with the large state space, we employ the deep Q-network to learn the Q-function that can help decide which cell is a better choice under a certain state during cell selection. Then, we modify the Q-network to a deep recurrent Q-network with LSTM to catch the temporal patterns and handle partial observability. Furthermore, we leverage the transfer learning techniques to relieve the dependency on a large amount of training data. Experiments on various real-life sensing datasets verify the effectiveness of our proposed algorithms over the state-of-the-art mechanisms in Sparse MCS by reducing up to 20% of sensed cells with the same data inference quality guarantee.
Document type :
Journal articles
Complete list of metadatas

Cited literature [31 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02321018
Contributor : Daqing Zhang <>
Submitted on : Wednesday, October 23, 2019 - 10:31:06 AM
Last modification on : Thursday, October 24, 2019 - 1:44:42 AM

File

6-Reinforcement Learning-based...
Explicit agreement for this submission

Identifiers

Collections

Citation

Wenbin Liu, Leye Wang, En Wang, Yongjian Yang, Djamal Zeghlache, et al.. Reinforcement learning-based cell selection in sparse mobile crowdsensing. Computer Networks, Elsevier, 2019, 161, pp.102-114. ⟨10.1016/j.comnet.2019.06.010⟩. ⟨hal-02321018⟩

Share

Metrics

Record views

21

Files downloads

67