Experiments with two-stage iterative solvers and preconditioned Krylov subspace methods on nearly completely decomposable Markov chains

Abstract

Preconditioned Krylov subspace methods are state-of-the-art iterative solvers developed mostly in the last fifteen years that may be used, among other things, to solve for the stationary distribution of Markov chains. Assuming Markov chains of interest are irreducible, the ¡problem amounts to computing a positive solution vector to a homogeneous system of linear algebraic equations with a singular coefficient matrix under a normalization constraint. That is, the (n X 1) unknown stationary vector x in Ax = 0, ||a:||^ = 1 (0.1 ) is sought. Here A = I — , an n x n singular M-matrix, and P is the one-step stochastic transition probability matrix. Albeit the recent advances, practicing performance analysts still widely prefer iterative methods based on splittings when they want to compare the performance of newly devised algorithms against existing ones, or when they need candidate solvers to evaluate the performance of a system model at hand. In fact, experimental results with Krylov subspace methods on Markov chains, especially the ill-conditioned nearly completely decomposable (NCD) ones, are few. We believe there is room for research in this area siDecifically to help us understand the effect of the degree of coupling of NCD Markov chains and their nonzero structure on the convergence characteristics and space requirements of preconditioned Krylov subspace methods. The work of several researchers have raised important and interesting questions that led to research in another, yet related direction. These questions are the following: “How must one go about partitioning the global coefficient matrix A in equation (0.1) into blocks if the system is NCD and a two-stage iterative solver (such as block successive overrelaxation— SOR) is to be employed? Are block partitionings dictated by the NCD normal form of F necessarily superior to others? Is it worth investing alternative partitionings? Better yet, for a fixed labelling and partitioning of the states, how does the performance of block SOR (or even that of point SOR) compare to the performance of the iterative aggregation-disaggregation (lAD) algorithm? Finally, is there any merit in using two-stage iterative solvers when preconditioned Krylov subspace methods are available?” Experimental results show that in most of the test cases two-stage iterative solvers are superior to Krylov subspace methods with the chosen preconditioners, on NCD Markov chains. For two-stage iterative solvers, there are cases in which a straightforward partitioning of the coefficient matrix gives a faster solution than can be obtained using the NCD normal form.