Computational methods for risk-averse undiscounted transient markov models
Author
Çavuş, O.
Ruszczyński, A.
Date
2014Source Title
Operations Research
Print ISSN
0030-364X
Electronic ISSN
1526-5463
Publisher
Institute for Operations Research and the Management Sciences (I N F O R M S)
Volume
62
Issue
2
Pages
401 - 417
Language
English
Type
ArticleItem Usage Stats
111
views
views
222
downloads
downloads
Abstract
The total cost problem for discrete-time controlled transient Markov models is considered. The objective functional is a Markov dynamic risk measure of the total cost. Two solution methods, value and policy iteration, are proposed, and their convergence is analyzed. In the policy iteration method, we propose two algorithms for policy evaluation: the nonsmooth Newton method and convex programming, and we prove their convergence. The results are illustrated on a credit limit control problem.
Keywords
Newton-Raphson methodControl problems
Cost problems
Dynamic risk measure
Markov model
Nonsmooth Newton method
Policy evaluation
Policy iteration
Solution methods
Markov processes