Selected Topics On Continuous-time Controlled Markov Chains And Markov Games (Icp Advanced Texts In Mathematics 5)

Selected Topics On Continuous-time Controlled Markov Chains And Markov Games (Icp Advanced Texts In Mathematics 5)

By: Onesimo Hernandez-Lerma (author), Tomas Prieto-Rumeau (author)Hardback

Up to 2 WeeksUsually despatched within 2 weeks


This book concerns continuous-time controlled Markov chains, also known as continuous-time Markov decision processes. They form a class of stochastic control problems in which a single decision-maker wishes to optimize a given objective function. This book is also concerned with Markov games, where two decision-makers (or players) try to optimize their own objective function. Both decision-making processes appear in a large number of applications in economics, operations research, engineering, and computer science, among other areas.An extensive, self-contained, up-to-date analysis of basic optimality criteria (such as discounted and average reward), and advanced optimality criteria (e.g., bias, overtaking, sensitive discount, and Blackwell optimality) is presented. A particular emphasis is made on the application of the results herein: algorithmic and computational issues are discussed, and applications to population models and epidemic processes are shown.This book is addressed to students and researchers in the fields of stochastic control and stochastic games. Moreover, it could be of interest also to undergraduate and beginning graduate students because the reader is not supposed to have a high mathematical background: a working knowledge of calculus, linear algebra, probability, and continuous-time Markov chains should suffice to understand the contents of the book.


Introduction; Controlled Markov Chains; Basic Optimality Criteria; Policy Iteration and Approximation Theorems; Overtaking, Bias, and Variance Optimality; Sensitive Discount Optimality; Blackwell Optimality; Constrained Controlled Markov Chains; Applications; Zero-Sum Markov Games; Bias and Overtaking Equilibria for Markov Games.

Product Details

  • ISBN13: 9781848168480
  • Format: Hardback
  • Number Of Pages: 292
  • ID: 9781848168480
  • ISBN10: 1848168489

Delivery Information

  • Saver Delivery: Yes
  • 1st Class Delivery: Yes
  • Courier Delivery: Yes
  • Store Delivery: Yes

Prices are for internet purchases only. Prices and availability in WHSmith Stores may vary significantly