Reinforcement learning for link adaptation and channel selection in leo satellite cognitive communications

Abstract

In this letter, we solve the link adaptation and channel selection problem in next generation satellite cognitive networks under dynamically varying channel availability and time-varying channel statistics. Primary user (PU) activity in Low Earth Orbit (LEO) satellite cognitive communications forces the set of available transmission channels for a secondary user (SU) to vary dynamically over time. We consider the scenario where the channel state varies in a piecewise-stationary mode, referred to as quasi-static (block-fading) channels. We formalize the problem as a reinforcement learning problem, and propose Discounted Structured and Sleeping Thompson Sampling (dSTS), which maximizes the SU’s throughput by selecting the optimum modulation and coding scheme (MCS) and the transmission channel under volatile and piecewise-stationary settings. When channel characteristics are unknown as well as piecewise-stationary, the proposed algorithm adapts the SU’s link-rate by exploiting the structure of the transmission success probability in transmission rates over the selected available channel. Furthermore, channel state information (CSI) is absent and feedback is limited to 1-bit (success/failure).