Markov Chains

1. Markov property, transition matrix, n-step transitions, Chapman-Kolmogorov equation.
2. Filtration, stopping times, strong Markov property, hitting times.
3. Communicating classes, closed sets, irreducibility, recurrence and transience, return times, absorption, aperiodicity.
4. Invariant measure and stationary distribtution, convergence theorem, ergodic theorem for Markov chains, positive and null recurrence.
5. Law of large numbers, time reversal, detailed balance. Examples: e.g. random walk, ruin problem, birth and death process, Galton Watson branching process, queuing model, Ehrenfest model.

After successful completion of the course, the student is capable to apply the Markov property, to analyze properties of Markov chains like irreducibility, aperiodicity and recurrence, to calculate stationary distributions and to apply the convergence theorem and the ergodic theorem.

MA0009 Introduction to Probability Theory and Statistics

The module is offered as lectures with accompanying practice sessions. In the lectures, the contents will be presented in a talk with demonstrative examples, as well as through discussion with the students. The lectures should animate the students to carry out their own analysis of the themes presented and to independently study the relevant literature. Corresponding to each lecture, practice sessions will be offered, in which exercise sheets and solutions will be available. In this way, students can deepen their understanding of the methods and concepts taught in the lectures and independently check their progress. At the beginning of the module, the practice sessions will be offered under guidance, but during the term the sessions will become more independent, and intensify learning individually as well as in small groups.

blackboard and computerized presentations

- Olle Häggström, Finite Markov chains and algorithmic applications, Cambridge University press, 2002.
- Norris, J.R. (1999) Markov Chains. Cambridge University Press.
- Wolfgang Woess, Denumerable Markov chains, European Mathematical Society, 2009.

The module examination is based on a written exam (60 minutes). Students show their ability to independently examine basic properties of Markov chains in limited time and apply them adequately. They have to calculate stationary distributions and apply the convergence theorem and the ergodic theorem.