de | en

Markov Chains

Module MA2404

This Module is offered by TUM Department of Mathematics.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Module version of WS 2011/2

There are historic module descriptions of this module. A module description is valid until replaced by a newer one.

available module versions
SS 2019WS 2011/2SS 2011

Basic Information

MA2404 is a semester module in German language at Bachelor’s level which is offered in winter semester.

This Module is included in the following catalogues within the study programs in physics.

  • Specialization Modules in Elite-Master Program Theoretical and Mathematical Physics (TMP)
Total workloadContact hoursCredits (ECTS)
150 h 45 h 5 CP

Content, Learning Outcome and Preconditions


1. Markov property, transition matrix, n-step transitions, Chapman-Kolmogorov equation.
2. Filtration, stopping times, strong Markov property, hitting times.
3. Communicating classes, closed sets, irreducibility, recurrence and transience, return times, absorption, aperiodicity.
4. Invariant measure and stationary distribtution, convergence theorem, ergodic theorem for Markov chains, positive and null recurrence.
5. Law of large numbers, time reversal, detailed balance. Examples: e.g. random walk, ruin problem, birth and death process, Galton Watson branching process, queuing model, Ehrenfest model.

Learning Outcome

After successful completion of the course, the student is capable to apply the Markov property, to analyze properties of Markov chains like irreducibility, aperiodicity and recurrence, to calculate stationary distributions and to apply the convergence theorem and the ergodic theorem.


MA0009 Introduction to Probability Theory and Statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

VO 2 Markov Chains Conache, D. Wed, 14:00–16:00, BC2 BC2 0.01.17
and singular or moved dates
UE 1 Markov Chains (Exercise Session) Conache, D. dates in groups

Learning and Teaching Methods

The module is offered as lectures with accompanying practice sessions. In the lectures, the contents will be presented in a talk with demonstrative examples, as well as through discussion with the students. The lectures should animate the students to carry out their own analysis of the themes presented and to independently study the relevant literature. Corresponding to each lecture, practice sessions will be offered, in which exercise sheets and solutions will be available. In this way, students can deepen their understanding of the methods and concepts taught in the lectures and independently check their progress. At the beginning of the module, the practice sessions will be offered under guidance, but during the term the sessions will become more independent, and intensify learning individually as well as in small groups.


blackboard and computerized presentations


- Olle Häggström, Finite Markov chains and algorithmic applications, Cambridge University press, 2002.
- Norris, J.R. (1999) Markov Chains. Cambridge University Press.
- Wolfgang Woess, Denumerable Markov chains, European Mathematical Society, 2009.

Module Exam

Description of exams and course work

The module examination is based on a written exam (60 minutes). Students show their ability to independently examine basic properties of Markov chains in limited time and apply them adequately. They have to calculate stationary distributions and apply the convergence theorem and the ergodic theorem.

Exam Repetition

The exam may be repeated at the end of the semester.

Top of page