Finite perturbation analysis methods for optimization of inventory systems with non-stationary Markov-modulated demand and partial information

The state of the economy may fluctuate due to several factors, and the customer demand is a affected from the fluctuations of the state of the economy. Although the inventory holders can predict the state of the economy based on the demand realizations, they generally do not have the true state information. The lack of information can be extended to the transition probabilities in the state, and the demand distributions associated with each state. Further extensions may include the actual number of demand states. We consider a single-item, periodic-review inventory system with Markov-modulated discrete-valued demand, constant lead time, and full backlogging. The true demand distribution state is partially observed based on the realized demands. We study the infinite horizon average cost minimization problem, in which the optimal inventory replenishment policy is a state-dependent base-stock policy. We develop a local search method based on finite perturbation analysis (FPA) to find the base-stock levels for a finite number of discretized state beliefs. We then extend our search method to the unknown transition matrix and demand distribution case. We compare the FPA-based local search algorithm with a myopic base-stock policy, the Viterbi algorithm, and the sufficient statistics method, in terms of the average cost. Finally, we analyze how the average cost changes with respect to the estimated number of demand states when the actual number of states is unknown.