Abstract.
Brown [3] constructed an aperiodic Markov decision chain in which no overtaking policy (stationary or nonstationary) exists. However, in his example a strong overtaking optimal policy exists in the class of all stationary policies. We provide another example of an aperiodic and geometric ergodic Markov decision chain which shows that the existence of a strong overtaking optimal stationary policy may fail inclusively in the class of stationary policies. We also give a brief survey of the existing literature on overtaking optimality in Markov decision chains.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Author information
Authors and Affiliations
Additional information
Manuscript received: December 1997/final version received: October 1998
Rights and permissions
About this article
Cite this article
Nowak, A., Vega-Amaya, O. A counterexample on overtaking optimality. Mathematical Methods of OR 49, 435–439 (1999). https://doi.org/10.1007/s001860050059
Issue Date:
DOI: https://doi.org/10.1007/s001860050059