Browsing by Subject "Markov chains"

Now showing 1 - 14 of 14

Open Access
Analyzing large sparse Markov chains of Kronecker products
(IEEE, 2009) Dayar, Tuğrul
Kronecker products are used to define the underlying Markov chain (MC) in various modeling formalisms, including compositional Markovian models, hierarchical Markovian models, and stochastic process algebras. The motivation behind using a Kronecker structured representation rather than a flat one is to alleviate the storage requirements associated with the MC. With this approach, systems that are an order of magnitude larger can be analyzed on the same platform. In the Kronecker based approach, the generator matrix underlying the MC is represented using Kronecker products [6] of smaller matrices and is never explicitly generated. The implementation of transient and steady-state solvers rests on this compact Kronecker representation, thanks to the existence of an efficient vector-Kronecker product multiplication algorithm known as the shuffle algorithm [6]. The transient distribution can be computed through uniformization using vector-Kronecker product multiplications. The steady-state distribution also needs to be computed using vector-Kronecker product multiplications, since direct methods based on complete factorizations, such as Gaussian elimination, normally introduce new nonzeros which cannot be accommodated. The two papers [2], [10] provide good overviews of iterative solution techniques for the analysis of MCs based on Kronecker products. Issues related to reachability analysis, vector-Kronecker product multiplication, hierarchical state space generation in Kronecker based matrix representations for large Markov models are surveyed in [5]. Throughout our discussion, we make the assumption that the MC at hand does not have unreachable states, meaning it is irreducible. And we take an algebraic view [7] to discuss recent results related to the analysis of MCs based on Kronecker products independently from modeling formalisms. We provide background material on the Kronecker representation of the generator matrix underlying a CTMC, show that it has a rich structure which is nested and recursive, and introduce a small CTMC whose generator matrix is expressed as a sum of Kronecker products; this CTMC is used as a running example throughout the discussion. We also consider preprocessing of the Kronecker representation so as to expedite numerical analysis. We discuss permuting the nonzero structure of the underlying CTMC symmetrically by reordering, changing the orders of the nested blocks by grouping, and reducing the size of the state space by lumping. The steady-state analysis of CTMCs based on Kronecker products is discussed for block iterative methods, multilevel methods, and preconditioned projection methods, respectively. The results can be extended to DTMCs based on Kronecker products with minor modifications. Areas that need further research are mentioned as they are discussed. Our contribution to this area over the years corresponds to work along iterative methods based on splittings and their block versions [11], associated preconditioners to be used with projection methods [4], near complete decomposability [8], a method based on iterative disaggregation for a class of lumpable MCs [9], a class of multilevel methods [3], and a recent method based on decomposition for weakly interacting subsystems [1]. © 2009 IEEE.
Open Access
Block SOR for Kronecker structured representations
(Elsevier, 2004) Buchholz, P.; Dayar, Tuğrul
The Kronecker structure of a hierarchical Markovian model (HMM) induces nested block partitionings in the transition matrix of its underlying Markov chain. This paper shows how sparse real Schur factors of certain diagonal blocks of a given partitioning induced by the Kronecker structure can be constructed from smaller component matrices and their real Schur factors. Furthermore, it shows how the column approximate minimum degree (COLAMD) ordering algorithm can be used to reduce fill-in of the remaining diagonal blocks that are sparse LU factorized. Combining these ideas, the paper proposes three-level block successive over-relaxation (BSOR) as a competitive steady state solver for HMMs. Finally, on a set of numerical experiments it demonstrates how these ideas reduce storage required by the factors of the diagonal blocks and improve solution time compared to an all LU factorization implementation of the BSOR solver. © 2004 Elsevier Inc. All rights reserved.
Open Access
Block SOR preconditioned projection methods for Kronecker structured Markovian representations
(SIAM, 2005) Buchholz, Peter; Dayar, Tuğrul
Kronecker structured representations are used to cope with the state space explosion problem in Markovian modeling and analysis. Currently, an open research problem is that of devising strong preconditioners to be used with projection methods for the computation of the stationary vector of Markov chains (MCs) underlying such representations. This paper proposes a block successive overrelaxation (BSOR) preconditioner for hierarchical Markovian models (HMMs1) that are composed of multiple low-level models and a high-level model that defines the interaction among low-level models. The Kronecker structure of an HMM yields nested block partitionings in its underlying continuous-time MC which may be used in the BSOR preconditioner. The computation of the BSOR preconditioned residual in each iteration of a preconditioned projection method becomes the problem of solving multiple nonsingular linear systems whose coefficient matrices are the diagonal blocks of the chosen partitioning. The proposed BSOR preconditioner solves these systems using sparse LU or real Schur factors of diagonal blocks. The fill-in of sparse LU factorized diagonal blocks is reduced using the column approximate minimum degree (COLAMD) ordering. A set of numerical experiments is presented to show the merits of the proposed BSOR preconditioner.
Open Access
Compact representation of solution vectors in Kronecker-based Markovian analysis
(Springer, 2016-08) Buchholz, P.; Dayar, Tuğrul; Kriege, J.; Orhan, M. Can
It is well known that the infinitesimal generator underlying a multi-dimensional Markov chain with a relatively large reachable state space can be represented compactly on a computer in the form of a block matrix in which each nonzero block is expressed as a sum of Kronecker products of smaller matrices. Nevertheless, solution vectors used in the analysis of such Kronecker-based Markovian representations still require memory proportional to the size of the reachable state space, and this becomes a bigger problem as the number of dimensions increases. The current paper shows that it is possible to use the hierarchical Tucker decomposition (HTD) to store the solution vectors during Kroneckerbased Markovian analysis relatively compactly and still carry out the basic operation of vector-matrix multiplication in Kronecker form relatively efficiently. Numerical experiments on two different problems of varying sizes indicate that larger memory savings are obtained with the HTD approach as the number of dimensions increases. © Springer International Publishing Switzerland 2016.
Open Access
Computing moments of first passage times to a subset of states in Markov chains
(SIAM, 2005) Dayar T.; Akar, N.
This paper presents a relatively efficient and accurate method to compute the moments of first passage times to a subset of states in finite ergodic Markov chains. With the proposed method, the moment computation problem is reduced to the solution of a linear system of equations with the right-hand side governed by a novel recurrence for computing the higher-order moments. We propose using a form of the Grassmann-Taksar-Heyman (GTH) algorithm to solve these linear equations. Due to the form of the linear systems involved, the proposed method does not suffer from the drawbacks associated with GTH in a row-wise sparse implementation. © 2005 Society for Industrial and Applied Mathematics.
Open Access
Conditional steady-state bounds for a subset of states in Markov chains
(ACM, 2006-10) Dayar, Tuğrul; Pekergin, N.; Younès, S.
The problem of computing bounds on the conditional steady-state probability vector of a subset of states in finite, ergodic discrete-time Markov chains (DTMCs) is considered. An improved algorithm utilizing the strong stochastic (st-)order is given. On standard benchmarks from the literature and other examples, it is shown that the proposed algorithm performs better than the existing one in the strong stochastic sense. Furthermore, in certain cases the conditional steady-state probability vector of the subset under consideration can be obtained exactly. Copyright 2006 ACM.
Open Access
Experiments with two-stage iterative solvers and preconditioned Krylov subspace methods on nearly completely decomposable Markov chains
(1997) Gueaieb, Wail
Preconditioned Krylov subspace methods are state-of-the-art iterative solvers developed mostly in the last fifteen years that may be used, among other things, to solve for the stationary distribution of Markov chains. Assuming Markov chains of interest are irreducible, the ¡problem amounts to computing a positive solution vector to a homogeneous system of linear algebraic equations with a singular coefficient matrix under a normalization constraint. That is, the (n X 1) unknown stationary vector x in Ax = 0, ||a:||^ = 1 (0.1 ) is sought. Here A = I — , an n x n singular M-matrix, and P is the one-step stochastic transition probability matrix. Albeit the recent advances, practicing performance analysts still widely prefer iterative methods based on splittings when they want to compare the performance of newly devised algorithms against existing ones, or when they need candidate solvers to evaluate the performance of a system model at hand. In fact, experimental results with Krylov subspace methods on Markov chains, especially the ill-conditioned nearly completely decomposable (NCD) ones, are few. We believe there is room for research in this area siDecifically to help us understand the effect of the degree of coupling of NCD Markov chains and their nonzero structure on the convergence characteristics and space requirements of preconditioned Krylov subspace methods. The work of several researchers have raised important and interesting questions that led to research in another, yet related direction. These questions are the following: “How must one go about partitioning the global coefficient matrix A in equation (0.1) into blocks if the system is NCD and a two-stage iterative solver (such as block successive overrelaxation— SOR) is to be employed? Are block partitionings dictated by the NCD normal form of F necessarily superior to others? Is it worth investing alternative partitionings? Better yet, for a fixed labelling and partitioning of the states, how does the performance of block SOR (or even that of point SOR) compare to the performance of the iterative aggregation-disaggregation (lAD) algorithm? Finally, is there any merit in using two-stage iterative solvers when preconditioned Krylov subspace methods are available?” Experimental results show that in most of the test cases two-stage iterative solvers are superior to Krylov subspace methods with the chosen preconditioners, on NCD Markov chains. For two-stage iterative solvers, there are cases in which a straightforward partitioning of the coefficient matrix gives a faster solution than can be obtained using the NCD normal form.
Open Access
Fixed point analysis of limited range share per node wavelength conversion in asynchronous optical packet switching systems
(Springer New York LLC, 2009) Akar, N.; Karasan, E.; Raffaelli, C.
In this article, we study an asynchronous optical packet switch equipped with a number of wavelength converters shared per node. The wavelength converters can be full range or circular-type limited range. We use the algorithmic methods devised for Markov chains of block-tridiagonal type in addition to fixed-point iterations to approximately solve this relatively complex system. In our approach, we also take into account the finite number of fiber interfaces using the Engset traffic model rather than the usual Poisson traffic modeling. The proposed analytical method provides an accurate approximation for full range systems for relatively large number of interfaces and for circular-type limited range wavelength conversion systems for which the tuning range is relatively narrow. © 2009 Springer Science+Business Media, LLC.
Open Access
On the convergence of a class of multilevel methods for large sparse Markov chains
(Society for Industrial and Applied Mathematics, 2007) Buchholz, P.; Dayar, T.
This paper investigates the theory behind the steady state analysis of large sparse Markov chains with a recently proposed class of multilevel methods using concepts from algebraic multigrid and iterative aggregation- disaggregation. The motivation is to better understand the convergence characteristics of the class of multilevel methods and to have a clearer formulation that will aid their implementation. In doing this, restriction (or aggregation) and prolongation (or disaggregation) operators of multigrid are used, and the Kronecker-based approach for hierarchical Markovian models is employed, since it suggests a natural and compact definition of grids (or levels). However, the formalism used to describe the class of multilevel methods for large sparse Markov chains has no influence on the theoretical results derived. © 2007 Society for Industrial and Applied Mathematics.
Open Access
Shared-per-wavelength asynchronous optical packet switching: a comparative analysis
(Elsevier, 2010-03-23) Akar, N.; Rafaelli, C.; Savi, M.; Karasan, E.
This paper compares four different architectures for sharing wavelength converters in asynchronous optical packet switches with variable-length packets. The first two architectures are the well-known shared-per-node (SPN) and shared-per-link (SPL) architectures, while the other two are the shared-per-input-wavelength (SPIW) architecture, recently proposed as an optical switch architecture in synchronous context only, which is extended here to the asynchronous scenario, and an original scheme called shared-per-output-wavelength (SPOW) architecture that we propose in the current article. We introduce novel analytical models to evaluate packet loss probabilities for SPIW and SPOW architectures in asynchronous context based on Markov chains and fixed-point iterations for the particular scenario of Poisson input traffic and exponentially distributed packet lengths. The models also account for unbalanced traffic whose impact is thoroughly studied. These models are validated by comparison with simulations which demonstrate that they are remarkably accurate. In terms of performance, the SPOW scheme provides blocking performance very close to the SPN scheme while maintaining almost the same complexity of the space switch, and employing less expensive wavelength converters. On the other hand, the SPIW scheme allows less complexity in terms of number of optical gates required, while it substantially outperforms the widely accepted SPL scheme. The authors therefore believe that the SPIW and SPOW schemes are promising alternatives to the conventional SPN and SPL schemes for the implementation of next-generation optical packet switching systems.
Open Access
SPM management using markov chain based data access prediction
(IEEE, 2008-11) Yemliha, T.; Srikantaiah, S.; Kandemir, M.; Öztürk, Özcan
Leveraging the power of scratchpad memories (SPMs) available in most embedded systems today is crucial to extract maximum performance from application programs. While regular accesses like scalar values and array expressions with affine subscript functions have been tractable for compiler analysis (to be prefetched into SPM), irregular accesses like pointer accesses and indexed array accesses have not been easily amenable for compiler analysis. This paper presents an SPM management technique using Markov chain based data access prediction for such irregular accesses. Our approach takes advantage of inherent, but hidden reuse in data accesses made by irregular references. We have implemented our proposed approach using an optimizing compiler. In this paper, we also present a thorough comparison of our different dynamic prediction schemes with other SPM management schemes. SPM management using our approaches produces 12.7% to 28.5% improvements in performance across a range of applications with both regular and irregular access patterns, with an average improvement of 20.8%.
Open Access
State aggregation-based model of asynchronous multi-fiber optical switching with shared wavelength converters
(Elsevier, 2013) Akar, N.; Raffaelli, C.; Savi, M.
This paper proposes new analytical models to study optical packet switching architectures with multi-fiber interfaces and shared wavelength converters. The multi-fiber extension of the recently proposed Shared-Per-Input-Wavelength (SPIW) scheme is compared against the multi-fiber Shared-Per-Node (SPN) scheme in terms of cost and performance for asynchronous traffic. In addition to using Markov chains and fixed-point iterations for modeling the mono-fiber case, a novel state aggregation technique is proposed to evaluate the packet loss in asynchronous multi-fiber scenario. The accuracy of the performance models is validated by comparison with simulations in a wide variety of scenarios with both balanced and imbalanced input traffic. The proposed analytical models are shown to remarkably capture the actual system behavior in all scenarios we tested. The adoption of multi-fiber interfaces is shown to achieve remarkable savings in the number of wavelength converters employed and their range. In addition, the SPIW solution allows to save, in particular conditions, a significant number of optical gates compared to the SPN solution. Indeed, SPIW allows, if properly dimensioned, potential complexity and cost reduction compared to SPN, while providing similar performance.
Open Access
Stochastic analysis of finite population bufferless multiplexing in optical packet/burst switching systems
(Denshi Jouhou Tsuushin Gakkai,Institute of Electronics Information and Communication Engineers, 2007) Akar, N.; Gunalay, Y.
In this letter, we study the blocking probabilities in an asynchronous optical packet/burst switching system with full wavelength conversion. Most of the existing work use Poisson traffic models that is well-suited for an infinite population of users. In this letter, the optical packet traffic arriving at the switching system is modeled through a superposition of a finite number of identical on-off sources. We propose a block tridiagonal LU factorization algorithm to efficiently solve the two dimensional Markov chain that arises in the modeling of the switching system.
Open Access
Stochastic comparison on nearly completely decomposable Markov chains
(2000) Alparslan, Denizhan N.
This thesis presents an improved version of a componentwise bounding algorithm for the steady state probability vector of nearly completely decomposable Markov chains. The given two-level algorithm uses aggregation and stochastic comparison with the strong stochastic (st) order. In order to improve accuracy, it employs reordering of states and a better componentwise probability bounding algorithm given st upper- and lower- bounding probability vectors. A thorough analysis of the algorithm is implemented in sparse storage and its implementation details are given. Numerical results on an application of wireless Asynchronous Transfer Mode network show that there are cases in which the given algorithm proves to be useful in computing bounds on the performance measures of the system. An improvement in the algorithm that must be considered to obtain better bounds on performance measures is also presented at the end.