Browsing by Subject "Markov Decision Processes"

Now showing 1 - 2 of 2

Open Access
Technical note-optimal structural results for assemble-to-order generalized M-Systems
(INFORMS Inst.for Operations Res.and the Management Sciences, 2014) Nadar, E.; Akan, M.; Scheller-Wolf, A.
We consider an assemble-to-order generalized M-system with multiple components and multiple products, batch ordering of components, random lead times, and lost sales. We model the system as an infinite-horizon Markov decision process and seek an optimal policy that specifies when a batch of components should be produced (i.e., inventory replenishment) and whether an arriving demand for each product should be satisfied (i.e., inventory allocation). We characterize optimal inventory replenishment and allocation policies under a mild condition on component batch sizes via a new type of policy: lattice-dependent base stock and lattice-dependent rationing. © 2014 INFORMS.
Open Access
Using reinforcement learning for dynamic link sharing problems under signaling constraints
(2003) Çelik, Nuri
In static link sharing system, users are assigned a fixed bandwidth share of the link capacity irrespective of whether these users are active or not. On the other hand, dynamic link sharing refers to the process of dynamically allocating bandwidth to each active user based on the instantaneous utilization of the link. As an example, dynamic link sharing combined with rate adaptation capability of multimedia applications provides a novel quality of service (QoS) framework for HFC and broadband wireless networks. Frequent adjustment of the allocated bandwidth in dynamic link sharing, yields a scalability issue in the form of a significant amount of message distribution and processing power (i.e. signaling) in the shared link system. On the other hand, if the rate of applications is adjusted once for the highest loaded traffic conditions, a significant amount of bandwidth may be wasted depending on the actual traffic load. There is then a need for an optimal dynamic link sharing system that takes into account the tradeoff between signaling scalability and bandwidth efficiency. In this work, we introduce a Markov decision framework for the dynamic link sharing system, when the desired signaling rate is imposed as a constraint. Reinforcement learning methodology is adopted for the solution of this Markov decision problem, and the results demonstrate that the proposed method provides better bandwidth efficiency without violating the signaling rate requirement compared to other heuristics.