Analysis of large Markov chains using stochastic automata networks

Abstract

This work contributes to the existing research in the area of analysis of finite Markov chains (MCs) modeled as stochastic automata networks (SANs). First, this thesis extends the near complete decomposability concept of Markov chains to SANs so that the inherent difficulty associated with solving the underlying MC car! be forecasted and solution techniques based on this concept car! be investigated. A straightforward approach to finding a nearly completely decomposable (NCD) partitioning of the MC underlying a SAN requires the computation of the nonzero elements of its global generator. This is not feasible for very large systems ever! in sparse matrix representation due to memory and execution time constraints. In this thesis, an efficient decompositional solution algorithm to this problem that is based on analyzing the NCD structure of each component of a giver! SAN is introduced. Numerical results show that the giver! algorithm performs much better than the straightforward approach. Second, this work specifies easy to check lumpability conditions for the generator of a SAN. When there exists a lumpable partitioning induced by the tensor representation of the generator, it is shown that an efficient iterative aggregation-disaggregation algorithm (IAD) may be employed to compute the steady state distribution of the MC underlying the SAN model. The results of experiments with continuous-time arid discrete-time SAN models show that the proposed algorithm performs better than the highly competitive block Gauss- Seidel (BGS) in terms of both the number of iterations arid the time to converge to the solution. having relatively large blocks in lurnpable partitionings is investigated. To overcome difficulties associated with solving large diagonal blocks at each iteration of the IAD algorithm, the recursive implementation of BGS for SANs is employed. The performance of IAD is compared with that of BGS. The results of experiments show that it is possible to tune IAD so that it outperforms BGS.