Markov Chains
Module MA2404
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
Module version of SS 2011
There are historic module descriptions of this module. A module description is valid until replaced by a newer one.
Whether the module’s courses are offered during a specific semester is listed in the section Courses, Learning and Teaching Methods and Literature below.
| available module versions | ||||
|---|---|---|---|---|
| WS 2021/2 | SS 2020 | SS 2019 | WS 2011/2 | SS 2011 |
Basic Information
MA2404 is a semester module in German language at Bachelor’s level which is offered in winter semester.
This Module is included in the following catalogues within the study programs in physics.
- Specialization Modules in Elite-Master Program Theoretical and Mathematical Physics (TMP)
| Total workload | Contact hours | Credits (ECTS) |
|---|---|---|
| 150 h | 45 h | 5 CP |
Content, Learning Outcome and Preconditions
Content
1. Markov property, transition matrix, n-step transitions, Chapman-Kolmogorov equation.
2. Filtration, stopping times, strong Markov property, hitting times.
3. Communicating classes, closed sets, irreducibility, recurrence and transience, return times, absorption, aperiodicity.
4. Invariant measure and stationary distribtution, convergence theorem, ergodic theorem for Markov chains, positive and null recurrence.
5. Law of large numbers, time reversal, detailed balance. Examples: e.g. random walk, ruin problem, birth and death process, Galton Watson branching process, queuing model, Ehrenfest model.
2. Filtration, stopping times, strong Markov property, hitting times.
3. Communicating classes, closed sets, irreducibility, recurrence and transience, return times, absorption, aperiodicity.
4. Invariant measure and stationary distribtution, convergence theorem, ergodic theorem for Markov chains, positive and null recurrence.
5. Law of large numbers, time reversal, detailed balance. Examples: e.g. random walk, ruin problem, birth and death process, Galton Watson branching process, queuing model, Ehrenfest model.
Learning Outcome
After successful completion of the course, the student is capable to apply the Markov property, to analyze properties of Markov chains like irreducibility, aperiodicity and recurrence, to calculate stationary distributions and to apply the convergence theorem and the ergodic theorem.
Preconditions
MA0009 Introduction to Probability Theory and Statistics
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
SS 2022
WS 2020/1
WS 2019/20
WS 2018/9
WS 2017/8
WS 2016/7
WS 2015/6
WS 2014/5
WS 2013/4
WS 2012/3
WS 2011/2
SS 2010
| Type | SWS | Title | Lecturer(s) | Dates | Links |
|---|---|---|---|---|---|
| VO | 2 | Markov Chains | |||
| UE | 1 | Markov Chains (Exercise Session) |
Learning and Teaching Methods
The module is offered as lectures with accompanying practice sessions. In the lectures, the contents will be presented in a talk with demonstrative examples, as well as through discussion with the students. The lectures should animate the students to carry out their own analysis of the themes presented and to independently study the relevant literature. Corresponding to each lecture, practice sessions will be offered, in which exercise sheets and solutions will be available. In this way, students can deepen their understanding of the methods and concepts taught in the lectures and independently check their progress. At the beginning of the module, the practice sessions will be offered under guidance, but during the term the sessions will become more independent, and intensify learning individually as well as in small groups.
Media
blackboard and computerized presentations
Literature
- Olle Häggström, Finite Markov chains and algorithmic applications, Cambridge University press, 2002.
- Norris, J.R. (1999) Markov Chains. Cambridge University Press.
- Wolfgang Woess, Denumerable Markov chains, European Mathematical Society, 2009.
- Norris, J.R. (1999) Markov Chains. Cambridge University Press.
- Wolfgang Woess, Denumerable Markov chains, European Mathematical Society, 2009.
Module Exam
Description of exams and course work
The module examination is based on a written exam (60 minutes). Students show their ability to independently examine basic properties of Markov chains in limited time and apply them adequately. They have to calculate stationary distributions and apply the convergence theorem and the ergodic theorem.
Exam Repetition
The exam may be repeated at the end of the semester.