Information Theory (Information Theory)

Module EI70350

This Module is offered by Chair of Communications Engineering (Prof. Kramer).

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Module version of SS 2020

There are historic module descriptions of this module. A module description is valid until replaced by a newer one.

Whether the module’s courses are offered during a specific semester is listed in the section Courses, Learning and Teaching Methods and Literature below.

available module versions
WS 2020/1	SS 2020

Basic Information

EI70350 is a semester module in English language at Master’s level which is offered in winter semester.

This Module is included in the following catalogues within the study programs in physics.

Catalogue of non-physics elective courses

Total workload	Contact hours	Credits (ECTS)
150 h	75 h	5 CP

Content, Learning Outcome and Preconditions

Content

Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity.
Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-Willems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channnels, Capacity. Relative entropy and additive white Gaussian noise channels.

Learning Outcome

After completion of the module the student is able to
• Explain the basic quantities of information theory, i.e., entropy, mutual information, informational divergence,
• Explain important properties of these quantities, e.g., chain rule, bounds, convexity
• Apply information theory to measure the quality of the processing blocks (e.g., data compression, channel coding) of a digital transmitter and receiver, and the quality of the channel (capacity)
• Understand scientific documents in information theory.

Preconditions

Basic principles of probability theory and statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

WS 2022/3 WS 2021/2 WS 2020/1 WS 2019/20 WS 2018/9 WS 2017/8 WS 2016/7

Type	SWS	Title	Lecturer(s)	Dates	Links

Learning and Teaching Methods

The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.
The students are instructed in an explorative teaching style. In the tutorials students discuss with the tutor exemplary implementation of codes introduced during the lecture, e.g. for current wirless mobile communictaion systems, and why certain coding schemes are used for certain applications.

Media

- Lecture notes
- Problem sheets
- Presentation slides

Literature

Further reading:
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition

Module Exam

Description of exams and course work

During a written exam (90 min) students proof their ability to apply principles of information theory (e.g., chain rule) by answering questions and apply information theory to measure the quality of the processing blocks (e.g., data compression, channel coding) of a digital transmitter and receiver as well as a mathematic methods for given problems, e.g. memoryless and stationary sources.

Exam Repetition

There is a possibility to take the exam in the following semester.

Current exam dates

Currently TUMonline lists the following exam dates. In addition to the general information above please refer to the current information given during the course.

Title
Time	Location	Info	Registration
Information Theory
	N1179	Import