de | en

# Markov Chains

## Module MA2404

This Module is offered by TUM Department of Mathematics.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

### Module version of SS 2019

There are historic module descriptions of this module. A module description is valid until replaced by a newer one.

Whether the module’s courses are offered during a specific semester is listed in the section Courses, Learning and Teaching Methods and Literature below.

available module versions
SS 2021SS 2020SS 2019WS 2011/2SS 2011

### Basic Information

MA2404 is a semester module in German language at Bachelor’s level which is offered in winter semester.

This Module is included in the following catalogues within the study programs in physics.

• Specialization Modules in Elite-Master Program Theoretical and Mathematical Physics (TMP)
150 h 45 h 5 CP

### Content, Learning Outcome and Preconditions

#### Content

1. Markov property, transition matrix, n-step transitions, Chapman-Kolmogorov equation.
2. Filtration, stopping times, strong Markov property, hitting times.
3. Communicating classes, closed sets, irreducibility, recurrence and transience, return times, absorption, aperiodicity.
4. Invariant measure and stationary distribtution, convergence theorem, ergodic theorem for Markov chains, positive and null recurrence.
5. Law of large numbers, time reversal, detailed balance. Examples: e.g. random walk, ruin problem, birth and death process, Galton Watson branching process, queuing model, Ehrenfest model.

#### Learning Outcome

After successful completion of the course, the student is capable to apply the Markov property, to analyze properties of Markov chains like irreducibility, aperiodicity and recurrence, to calculate stationary distributions and to apply the convergence theorem and the ergodic theorem.

#### Preconditions

MA0009 Introduction to Probability Theory and Statistics

### Courses, Learning and Teaching Methods and Literature

#### Learning and Teaching Methods

The module is offered as lectures with accompanying practice sessions. In the lectures, the contents will be presented in a talk with demonstrative examples, as well as through discussion with the students. The lectures should animate the students to carry out their own analysis of the themes presented and to independently study the relevant literature. Corresponding to each lecture, practice sessions will be offered, in which exercise sheets and solutions will be available. In this way, students can deepen their understanding of the methods and concepts taught in the lectures and independently check their progress. At the beginning of the module, the practice sessions will be offered under guidance, but during the term the sessions will become more independent, and intensify learning individually as well as in small groups.

#### Media

blackboard and computerized presentations

#### Literature

- Olle Häggström, Finite Markov chains and algorithmic applications, Cambridge University press, 2002.
- Norris, J.R. (1999) Markov Chains. Cambridge University Press.
- Wolfgang Woess, Denumerable Markov chains, European Mathematical Society, 2009.

### Module Exam

#### Description of exams and course work

Aktueller Hinweis angesichts des eingeschränkten Präsenzbetriebs auf Grund der CoViD19-Pandemie: Die Prüfungsform ändert sich gemäß §13a APSO einmalig auf das Prüfungsformat: einmalige Übungsleistung.
In dieser wird überprüft, inwieweit die Studierenden die grundlegenden Eigenschaften von Markovketten in begrenzter Zeit eigenständig untersuchen und angemessen anwenden können sowie stationäre Verteilungen berechnen und den Konvergenzsatz und den Ergodensatz anwenden können.

#### Exam Repetition

The exam may be repeated at the end of the semester.

#### Current exam dates

Currently TUMonline lists the following exam dates. In addition to the general information above please refer to the current information given during the course.

Title
TimeLocationInfoRegistration
Markov Chains
Mon, 2021-03-01, 9:00 till 10:00 Einmalige Übungsleistung /one-time electronic exercise performances till 2021-01-15 (cancelation of registration till 2021-02-22)
Tue, 2021-03-30, 11:30 till 12:30 Einmalige Übungsleistung / one-time electronic exercise performances till 2021-03-22 (cancelation of registration till 2021-03-23)
Top of page