Fully distributed bandit algorithm for the joint channel and rate selection problem in heterogeneous cognitive radio networks

We consider the problem of the distributed sequential channel and rate selection in cognitive radio networks where multiple users choose channels from the same set of available wireless channels and pick modulation and coding schemes (corresponds to transmission rates). In order to maximize the network throughput, users need to be cooperative while communication among them is not allowed. Also, if multiple users select the same channel simultaneously, they collide, and none of them would be able to use the channel for transmission. We rigorously formulate this resource allocation problem as a multi-player multi-armed bandit problem and propose a decentralized learning algorithm called Game of Thrones with Sequential Halving Orthogonal Exploration (GoT-SHOE). The proposed algorithm keeps the number of collisions in the network as low as possible and performs almost optimal exploration of the transmission rates to speed up the learning process. We prove our learning algorithm achieves a regret with respect to the optimal allocation that grows logarithmically over rounds with a leading term that is logarithmic in the number of transmission rates. We also propose an extension of our algorithm which works when the number of users is greater than the number of channels. Moreover, we discuss that Sequential Halving Orthogonal Exploration can indeed be used with any distributed channel assignment algorithm and enhance its performance. Finally, we provide extensive simulations and compare the performance of our learning algorithm with the state-of-the-art which demonstrates the superiority of the proposed algorithm in terms of better system throughput and lower number of collisions.