Browsing by Subject "Markov Chains"

Now showing 1 - 7 of 7

Open Access
Comparison of multilevel methods for kronecker-based Markovian representations
(Springer, 2004) Buchholz, P.; Dayar T.
The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates.
Open Access
Cost-effectiveness of adjuvant FOLFOX and 5FU/LV chemotherapy for patients with stage II colon cancer
(2013) Ayvaci, M.U.S.; Shi J.; Alagoz O.; Lubner, S.J.
Purpose. We evaluated the cost-effectiveness of adjuvant chemotherapy using 5-fluorouracil, leucovorin (5FU/LV), and oxaliplatin (FOLFOX) compared with 5FU/LV alone and 5FU/LV compared with observation alone for patients who had resected stage II colon cancer. Methods. We developed 2 Markov models to represent the adjuvant chemotherapy and follow-up periods and a single Markov model to represent the observation group. We used calibration to estimate the transition probabilities among different toxicity levels. The base case considered 60-year-old patients who had undergone an uncomplicated hemicolectomy for stage II colon cancer and were medically fit to receive 6 months of adjuvant chemotherapy. We measured health outcomes in quality-adjusted life-years (QALYs) and estimated costs using 2007 US dollars. Results. In the base case, adjuvant chemotherapy of the FOLFOX regimen had an incremental cost-effectiveness ratio (ICER) of $54,359/QALY compared with the 5FU/LV regimen, and the 5FU/LV regimen had an ICER of $14,584/QALY compared with the observation group from the third-party payer perspective. The ICER values were most sensitive to 5-year relapse probability, cost of adjuvant chemotherapy, and the discount rate for the FOLFOX arm, whereas the ICER value of 5FU/LV was most sensitive to the 5-year relapse probability, 5-year survival probability, and the relapse cost. The probabilistic sensitivity analysis indicates that the ICER of 5FU/LV is less than $50,000/QALY with a probability of 99.62%, and the ICER of FOLFOX as compared with 5FU/LV is less than $50,000/QALY and $100,000/QALY with a probability of 44.48% and 97.24%, respectively. Conclusion. Although adjuvant chemotherapy with 5FU/LV is cost-effective at all ages for patients who have undergone an uncomplicated hemicolectomy for stage II colon cancer, FOLFOX is not likely to be cost-effective as compared with 5FU/LV.
Open Access
On the effects of using the Grassmann-Taksar-Heyman method in iterative aggregation-disaggregation
(SIAM, 1996) Dayar T.; Stewart, W. J.
Iterative aggregation-disaggregation (IAD) is an effective method for solving finite nearly completely decomposable (NCD) Markov chains. Small perturbations in the transition probabilities of these chains may lead to considerable changes in the stationary probabilities; NCD Markov chains are known to be ill-conditioned. During an IAD step, this undesirable condition is inherited by the coupling matrix and one confronts the problem of finding the stationary probabilities of a stochastic matrix whose diagonal elements are close to 1. In this paper, the effects of using the Grassmann-Taksar-Heyman (GTH) method to solve the coupling matrix formed in the aggregation step are investigated. Then the idea is extended in such a way that the same direct method can be incorporated into the disaggregation step. Finally, the effects of using the GTH method in the IAD algorithm on various examples are demonstrated, and the conditions under which it should be employed are explained.
Open Access
Quasi lumpability, lower-bounding coupling matrices, and nearly completely decomposable Markov chains
(SIAM, 1997) Dayar T.; Stewart, W. J.
In this paper, it is shown that nearly completely decomposable (NCD) Markov chains are quasi-lumpable. The state space partition is the natural one, and the technique may be used to compute lower and upper bounds on the stationary probability of each NCD block. In doing so, a lower-bounding nonnegative coupling matrix is employed. The nature of the stationary probability bounds is closely related to the structure of this lower-bounding matrix. Irreducible lower-bounding matrices give tighter bounds compared with bounds obtained using reducible lower-bounding matrices. It is also noticed that the quasi-lumped chain of an NCD Markov chain is an ill-conditioned matrix and the bounds obtained generally will not be tight. However, under some circumstances, it is possible to compute the stationary probabilities of some NCD blocks exactly.
Open Access
Quasi-birth-and-death processes with level-geometric distribution
(SIAM, 2003) Dayar T.; Quessette, F.
A special class of homogeneous continuous-time quasi-birth-and-death (QBD) Markov chains (MCS) which possess level-geometric (LG) stationary distribution is considered. Assuming that the stationary vector is partitioned by levels into subvectors, in an LG distribution all stationary subvectors beyond a finite level number are multiples of each other. Specifically, each pair of stationary subvectors that belong to consecutive levels is related by the same scalar, hence the term level-geometric. Necessary and sufficient conditions are specified for the existence of such a distribution, and the results are elaborated in three examples.
Open Access
State space orderings for Gauss-Seidel in Markov chains revisited
(SIAM, 1998) Dayar, Tuğrul
States of a Markov chain may be reordered to reduce the magnitude of the subdominant eigenvalue of the Gauss-Seidel (GS) iteration matrix. Orderings that maximize the elemental mass or the number of nonzero elements in the dominant term of the GS splitting (that is, the term approximating the coefficient matrix) do not necessarily converge faster. An ordering of a Markov chain that satisfies Property-R is semiconvergent. On the other hand, there are semiconvergent state space orderings that do not satisfy Property-R. For a given ordering, a simple approach for checking Property-R is shown. Moreover, a version of the Cuthill-McKee algorithm may be used to order the states of a Markov chain so that Property-R is satisfied. The computational complexity of the ordering algorithm is less than that of a single GS iteration. In doing all this, the aim is to gain insight into (faster) converging orderings.
Open Access
Stochastic automata networks and near complete decomposability
(SIAM, 2002) Gusak, O.; Dayar T.; Fourneau, J. M.
Stochastic automata networks (SANs) have been developed and used in the last fifteen years as a modeling formalism for large systems that can be decomposed into loosely connected components. In this work, we extend the near complete decomposability concept of Markov chains (MCs) to SANs so that the inherent difficulty associated with solving the underlying MC can be forecasted and solution techniques based on this concept can be investigated. A straightforward approach to finding a nearly completely decomposable (NCD) partitioning of the MC underlying a SAN requires the computation of the nonzero elements of its global generator. This is not feasible for very large systems even in sparse matrix representation due to memory and execution time constraints. We devise an efficient decompositional solution algorithm to this problem that is based on analyzing the NCD structure of each component of a given SAN. Numerical results show that the given algorithm performs much better than the straightforward approach.