Browsing by Subject "Markov chain"

Now showing 1 - 13 of 13

Open Access
Analysis of large Markov chains using stochastic automata networks
(2001-07) Oleg, Gusak
This work contributes to the existing research in the area of analysis of finite Markov chains (MCs) modeled as stochastic automata networks (SANs). First, this thesis extends the near complete decomposability concept of Markov chains to SANs so that the inherent difficulty associated with solving the underlying MC car! be forecasted and solution techniques based on this concept car! be investigated. A straightforward approach to finding a nearly completely decomposable (NCD) partitioning of the MC underlying a SAN requires the computation of the nonzero elements of its global generator. This is not feasible for very large systems ever! in sparse matrix representation due to memory and execution time constraints. In this thesis, an efficient decompositional solution algorithm to this problem that is based on analyzing the NCD structure of each component of a giver! SAN is introduced. Numerical results show that the giver! algorithm performs much better than the straightforward approach. Second, this work specifies easy to check lumpability conditions for the generator of a SAN. When there exists a lumpable partitioning induced by the tensor representation of the generator, it is shown that an efficient iterative aggregation-disaggregation algorithm (IAD) may be employed to compute the steady state distribution of the MC underlying the SAN model. The results of experiments with continuous-time arid discrete-time SAN models show that the proposed algorithm performs better than the highly competitive block Gauss- Seidel (BGS) in terms of both the number of iterations arid the time to converge to the solution. having relatively large blocks in lurnpable partitionings is investigated. To overcome difficulties associated with solving large diagonal blocks at each iteration of the IAD algorithm, the recursive implementation of BGS for SANs is employed. The performance of IAD is compared with that of BGS. The results of experiments show that it is possible to tune IAD so that it outperforms BGS.
Open Access
Automatic detection of compound structures by joint selection of region groups from a hierarchical segmentation
(Institute of Electrical and Electronics Engineers, 2016) Akçay, H. G.; Aksoy, S.
A challenging problem in remote sensing image analysis is the detection of heterogeneous compound structures such as different types of residential, industrial, and agricultural areas that are composed of spatial arrangements of simple primitive objects such as buildings and trees. We describe a generic method for the modeling and detection of compound structures that involve arrangements of an unknown number of primitives in large scenes. The modeling process starts with a single example structure, considers the primitive objects as random variables, builds a contextual model of their arrangements using a Markov random field, and learns the parameters of this model via sampling from the corresponding maximum entropy distribution. The detection task is formulated as the selection of multiple subsets of candidate regions from a hierarchical segmentation where each set of selected regions constitutes an instance of the example compound structure. The combinatorial selection problem is solved by the joint sampling of groups of regions by maximizing the likelihood of their individual appearances and relative spatial arrangements. Experiments using very high spatial resolution images show that the proposed method can effectively localize an unknown number of instances of different compound structures that cannot be detected by using spectral and shape features alone.
Open Access
Conditions for uniqueness of limit Gibbs states
(1998) Şahin, Mehmet Arafat
In this work we studied the problem of phase transitions in one-dirnensional models with unique ground state. A model ha\dng two spins, one ground state and exhibiting phase transition is constructed.
Open Access
Decompositional analysis of Kronecker structured Markov chains
(Kent State University, 2008) Bao, Y.; Bozkur, I. N.; Dayar, T.; Sun, X.; Trivedi, K. S.
This contribution proposes a decompositional iterative method with low memory requirements for the steadystate analysis ofKronecker structured Markov chains. The Markovian system is formed by a composition of subsystems using the Kronecker sum operator for local transitions and the Kronecker product operator for synchronized transitions. Even though the interactions among subsystems, which are captured by synchronized transitions, need not be weak, numerical experiments indicate that the solver benefits considerably from weak interactions among subsystems, and is to be recommended specifically in this case. © 2008, Kent State University.
Open Access
Economic design of EWMA control charts based on loss function
(Elsevier, 2009) Serel, D. A.
For monitoring the stability of a process, various control charts based on exponentially weighted moving average (EWMA) statistics have been proposed in the literature. We study the economic design of EWMA-based mean and dispersion charts when a linear, quadratic, or exponential loss function is used for computing the costs arising from poor quality. The chart parameters (sample size, sampling interval, control limits and smoothing constant) minimizing the overall cost of the control scheme are determined via computational methods. Using numerical examples, we compare the performances of the EWMA charts with Shewhart over(X, -) and S charts, and investigate the sensitivity of the chart parameters to changes in process parameters and loss functions. Numerical results imply that rather than sample size or control limits, the users need to adjust the sampling interval in response to changes in the cost of poor quality.
Open Access
Non-stationary Markov chains
(1996) Mallak, Saed
In thi.s work, we studierl the Ergodicilv of Non-Stationary .Markov chains. We gave several e.xainples with different cases. We proved that given a sec[uence of Markov chains such that the limit of this sec|uence is an Ergodic Markov chain, then the limit of the combination of the elements of this sequence is again Ergodic (under some condition if the state space is infinite). We also proved that the limit of the combination of an arbitrary sequence of Markov chains on a finite state space is Weak Ergodic if it satisfies some condition. Under the same condition, the limit of the combination of a doubly stochastic sequence of Markov chains is Ergodic.
Open Access
On compact solution vectors in Kronecker-based Markovian analysis
(Elsevier, 2017) Buchholz, P.; Dayar T.; Kriege, J.; Orhan, M. C.
State based analysis of stochastic models for performance and dependability often requires the computation of the stationary distribution of a multidimensional continuous-time Markov chain (CTMC). The infinitesimal generator underlying a multidimensional CTMC with a large reachable state space can be represented compactly in the form of a block matrix in which each nonzero block is expressed as a sum of Kronecker products of smaller matrices. However, solution vectors used in the analysis of such Kronecker-based Markovian representations require memory proportional to the size of the reachable state space. This implies that memory allocated to solution vectors becomes a bottleneck as the size of the reachable state space increases. Here, it is shown that the hierarchical Tucker decomposition (HTD) can be used with adaptive truncation strategies to store the solution vectors during Kronecker-based Markovian analysis compactly and still carry out the basic operations including vector–matrix multiplication in Kronecker form within Power, Jacobi, and Generalized Minimal Residual methods. Numerical experiments on multidimensional problems of varying sizes indicate that larger memory savings are obtained with the HTD approach as the number of dimensions increases. © 2017 Elsevier B.V.
Open Access
On the numerical analysis of infinite multi-dimensional Markov chains
(2017-07) Orhan, Muhsin Can
A system with multiple interacting subsystems that exhibits the Markov property can be represented as a multi-dimensional Markov chain (MC). Usually the reachable state space of this MC is a proper subset of its product state space, that is, Cartesian product of its subsystem state spaces. Compact storage of the infinitesimal generator matrix underlying such a MC and efficient implementation of analysis methods using Kronecker operations require the set of reachable states to be represented as a union of Cartesian products of subsets of subsystem state spaces. We first show that the problem of partitioning the reachable state space of a three or higher dimensional system with a minimum number of partitions into Cartesian products of subsets of subsystem state spaces is NP-complete. Two algorithms, one merge based the other refinement based, that yield possibly nonoptimal partitionings are presented. Results of experiments on a set of problems from the literature and those that are randomly generated indicate that, although it may be more time and memory consuming, the refinement based algorithm almost always computes partitionings with a smaller number of partitions than the merge based algorithm. When the infinitesimal generator matrix underlying the MC is represented compactly using Kronecker products, analysis methods based on vector– Kronecker product multiplication need to be employed. When the factors in the Kronecker product terms are relatively dense, vector–Kronecker product multiplication can be performed efficiently by the shuffle algorithm. When the factors are relatively sparse, it may be more efficient to obtain nonzero elements of the generator matrix in Kronecker form on-the-fly and multiply them with corresponding elements of the vector. We propose a modification to the shuffle algorithm that multiplies relevant elements of the vector with submatrices of factors in which zero rows and columns are omitted. This approach avoids unnecessary floating-point operations that evaluate to zero during the course of the multiplication. Numerical experiments on a large number of models indicate that, in many cases the modified shuffle algorithm performs a smaller number of floating-point operations than the shuffle algorithm and the algorithm that generates nonzeros on-the-fly, sometimes with minimum number of floating-point operations and amount of memory possible. Although the generator matrix is stored compactly using Kronecker products, solution vectors used in the analysis still require memory proportional to the size of the reachable state space. This becomes a bigger problem as the number of dimensions increases. We show that it is possible to use the hierarchical Tucker decomposition (HTD) to store the solution vectors during Kroneckerbased Markovian analysis relatively compactly and still carry out the basic operation of vector–matrix multiplication in Kronecker form relatively efficiently. The time evolution of a stochastic chemical system modelled as a continuoustime MC (CTMC) can be described as a system of ordinary differential equations (ODEs) known as the chemical master equation (CME). The CME can be analyzed by discretizing time and solving a linear system obtained by truncating the countably infinite state space at each time step. However, it is not trivial to choose a truncated state space that includes few states with negligible probabilities and leaves out only a small probability mass. We show that it is possible to decrease the memory requirement of the ODE solver using HTD with adaptive truncation strategies and we propose a novel approach to truncate the countably infinite state space using prediction vectors. Numerical experiments indicate that adaptive truncation strategies improve time and memory efficiency significantly when fixed truncation strategies are inefficient. Finally, we consider a multi-class multi-server retrial queueing system in which customers arrive according to a class-dependent Markovian arrival process (MAP). Service and retrial times follow class-dependent phase-type (PH) distributions with the further assumption that PH distributions of retrial times are acyclic. Here, we obtain a necessary and sufficient condition for ergodicity from criteria based on drifts. The countably infinite state space of the model is truncated with an appropriately chosen Lyapunov function. The truncated model is described as a multi-dimensional MC and a Kronecker representation of its infinitesimal generator matrix is numerically analyzed.
Open Access
On the numerical solution of Kronecker-based infinite level-dependent QBD processes
(2013) Baumann, H.; Dayar, T.; Orhan, M. C.; Sandmann, W.
Infinite level-dependent quasi-birth-and-death (LDQBD) processes can be used to model Markovian systems with countably infinite multidimensional state spaces. Recently it has been shown that sums of Kronecker products can be used to represent the nonzero blocks of the transition rate matrix underlying an LDQBD process for models from stochastic chemical kinetics. This paper extends the form of the transition rates used recently so that a larger class of models including those of call centers can be analyzed for their steady-state. The challenge in the matrix analytic solution then is to compute conditional expected sojourn time matrices of the LDQBD model under low memory and time requirements after truncating its countably infinite state space judiciously. Results of numerical experiments are presented using a Kronecker-based matrix-analytic solution on models with two or more countably infinite dimensions and rules of thumb regarding better implementations are derived. In doing this, a more recent approach that reduces memory requirements further by enabling the computation of steady-state expectations without having to obtain the steady-state distribution is also considered. © 2013 Elsevier B.V. All rights reserved.
Open Access
On vector-kronecker product multiplication with rectangular factors
(Society for Industrial and Applied Mathematics, 2015) Dayar, T.; Orhan, M. C.
The infinitesimal generator matrix underlying a multidimensional Markov chain can be represented compactly by using sums of Kronecker products of small rectangular matrices. For such compact representations, analysis methods based on vector-Kronecker product multiplication need to be employed. When the factors in the Kronecker product terms are relatively dense, vector- Kronecker product multiplication can be performed efficiently by the shuffle algorithm. When the factors are relatively sparse, it may be more efficient to obtain nonzero elements of the generator matrix in Kronecker form on the fly and multiply them with corresponding elements of the vector. This work proposes a modification to the shuffle algorithm that multiplies relevant elements of the vector with submatrices of factors in which zero rows and columns are omitted. This approach avoids unnecessary floating-point operations that evaluate to zero during the course of the multiplication and possibly reduces the amount of memory used. Numerical experiments on a large number of models indicate that in many cases the modified shuffle algorithm performs a smaller number of floating-point operations than the shuffle algorithm and the algorithm that generates nonzeros on the fly, sometimes with a minimum number of floating-point operations and as little of memory possible.
Open Access
Oscillation properties of expected stopping times and stopping probabilities for patterns consisting of consecutive states in Markov chains
(Rocky Mountain Mathematics Consortium, 2020) Kerimov, Azer; Öner, Abdullah
We investigate a Markov chain with a state space 1,2,…,r1,2,…,r stopping at appearance of patterns consisting of two consecutive states. It is observed that the expected stopping times of the chain have surprising oscillating dependencies on starting positions. Analogously, the stopping probabilities also have oscillating dependencies on terminal states. In a nonstopping Markov chain the frequencies of appearances of two consecutive states are found explicitly.
Open Access
Phase transition in one dimensional model with unique ground state
(Elsevier BV * North-Holland, 1996) Kerimov, A.
A one - dimensional model having a unique ground state and admitting a phase transition is constructed.
Open Access
Steady-state analysis of a multiclass MAP/PH/c queue with acyclic PH retrials
(Applied Probability Trust, 2016) Dayar T.; Orhan, M. C.
A multiclass c-server retrial queueing system in which customers arrive according to a class-dependent Markovian arrival process (MAP) is considered. Service and retrial times follow class-dependent phase-type (PH) distributions with the further assumption that PH distributions of retrial times are acyclic. A necessary and sufficient condition for ergodicity is obtained from criteria based on drifts. The infinite state space of the model is truncated with an appropriately chosen Lyapunov function. The truncated model is described as a multidimensional Markov chain, and a Kronecker representation of its generator matrix is numerically analyzed.