Markov Chains: Theory and Applications (Applied Stochastic Methods)
Bruno Sericola
Synopsis "Markov Chains: Theory and Applications (Applied Stochastic Methods) "
Markov chains are a fundamental class of stochastic processes.They are widely used to solve problems in a large number of domainssuch as operational research, computer science, communicationnetworks and manufacturing systems. The success of Markov chains ismainly due to their simplicity of use, the large number ofavailable theoretical results and the quality of algorithmsdeveloped for the numerical evaluation of many metrics ofinterest.The author presents the theory of both discrete-time andcontinuous-time homogeneous Markov chains. He carefully examinesthe explosion phenomenon, the Kolmogorov equations, the convergenceto equilibrium and the passage time distributions to a state and toa subset of states. These results are applied to birth-and-deathprocesses. He then proposes a detailed study of the uniformizationtechnique by means of Banach algebra. This technique is used forthe transient analysis of several queuing systems.Contents1. Discrete-Time Markov Chains2. Continuous-Time Markov Chains3. Birth-and-Death Processes4. Uniformization5. QueuesAbout the AuthorsBruno Sericola is a Senior Research Scientist at Inria Rennes– Bretagne Atlantique in France. His main research activityis in performance evaluation of computer and communication systems,dependability analysis of fault-tolerant systems and stochasticmodels.