Browsing by Subject "Markov decision processes (MDPs)"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemOpen Access
    Water-filling-based scheduling for weighted binary freshness in cache update systems
    (Institute of Electrical and Electronics Engineers, 2024-03-01) Gamgam, Ege Orkun; Akar, Nail
    We consider a cache update system with a remote server delivering time-varying contents of multiple Internet of Things (IoT) items with heterogeneous popularities and service times to a local cache so as to keep the items as fresh as possible at the cache. New content for an item arrives at the server according to a Poisson process and the server is equipped with multiple queues each of which holds the most up-to-date content for the corresponding item. In this setting, we study several scheduling policies employed at the server so as to maximize the popularity-weighted binary freshness across the items. The scheduling problem is first formulated as an infinite-horizon average-reward Markov decision process (MDP) which suffers from the curse of dimensionality when the number of items is large. We then propose a water-filling-based scheduling (WFS) policy and its extension, namely, extended WFS (E-WFS) policy, with worst case complexities being quadratic and cubic in the number of items, respectively, based on convex optimization applied to a relaxation of the original system. Simulation results are provided to validate the effectiveness of the proposed policies.