Approximate dynamic programming approach for sequential change diagnosis problem

Abstract

We study sequential change diagnosis problem which is the combination of change diagnosis and multi-hypothesis testing problem. One observes a sequence of independent and identically distributed random variables. At a sudden disorder time, the probability distribution of the random variables change. The disorder time and its cause are unavailable to the observer. The problem is to detect this abrupt change in the distribution of the random process as quickly as possible and identify its cause as accurately as possible. Dayanık et al. [Dayanık, Goulding and Poor, Bayesian sequential change diagnosis, Mathematics of Operations Research, vol. 45, pp. 475-496, 2008] reduce the problem to a Markov optimal stopping problem and provide an optimal sequential decision strategy. However, only a small subset of the problems is computationally feasible due to curse of dimensionality. The subject of this thesis is to search for the means to overcome the curse of dimensionality. To this end, we propose several approximate dynamic programming algorithms to solve large change diagnosis problems. On several numerical examples, we compare their performance against the performance of optimal dynamic programming solution.