Information Theory

Module EI5005

This Module is offered by TUM Department of Electrical and Computer Engineering.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

EI5005 is a semester module in English language at Master’s level which is offered in winter semester.

This module description is valid to SS 2013.

Total workloadContact hoursCredits (ECTS)
90 h 45 h 3 CP

Content, Learning Outcome and Preconditions


Review of probability theory. Uncertainty and mutual information. Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Arithmetic codes, Lempel-Ziv codes, Elias-Willems codes. Channel coding: Discrete Memoryless Channnels, Capacity, Fano's inequality, Random Coding. Additive white Gaussian noise channels.

Learning Outcome

The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.


Basic principles of probability theory and statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

VU 3 Information Theory and Source Coding Dienstag, 08:45–09:30
Dienstag, 15:00–16:30

Learning and Teaching Methods

Lerning method: Lectures, tutorials, excercises, individual study. Teaching method: The students are instructed in an explorative teaching style.


- Lecture notes - Problem sheets - Presentation slides


Further reading: - Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition

Module Exam

Description of exams and course work

Examination with the following elements: - Written examination

Exam Repetition

There is a possibility to take the exam in the following semester.

Condensed Matter

When atoms interact things can get interesting. Fundamental research on the underlying properties of materials and nanostructures and exploration of the potential they provide for applications.

Nuclei, Particles, Astrophysics

A journey of discovery to understanding our world at the subatomic scale, from the nuclei inside atoms down to the most elementary building blocks of matter. Are you ready for the adventure?


Biological systems, from proteins to living cells and organisms, obey physical principles. Our research groups in biophysics shape one of Germany's largest scientific clusters in this area.