Markov decision processes with unbounded transition rates: structural properties of the relative value function
Summary
In this thesis we have studied Markov decision processes with unbounded transition rates. In
order to facilitate the search for optimal policies, we are interested in structural properties of
the relative value function of these systems. Properties of interest are for example monotonicity
or convexity as a function of the input parameters. This can not be done by the standard
mathematical tools, since the systems are not uniformizable. In this study we have examined
whether a newly developed method called Smoothed Rate Truncation can overcome this problem.
We introduce how this method is used for a processor sharing queue. We have shown that it can
be applied to a system with service control. We have also obtained nice results in the framework of
event-based dynamic programming. Due to Smoothed Rate Truncation new operators arise. We
have shown that for these operators propagation results, similar to results for existing operators,
can be derived. We can conclude that Smoothed Rate Truncation can be used to analyse other
processes that have unbounded transition rates.