de | en

Information Theory

Module EI7431

This Module is offered by TUM Department of Electrical and Computer Engineering.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

EI7431 is a semester module in English language at Master’s level which is offered in winter semester.

This module description is valid from WS 2014/5 to SS 2016.

Total workloadContact hoursCredits (ECTS)
150 h 60 h 5 CP

Content, Learning Outcome and Preconditions


Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity.
Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-Willems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channnels, Capacity. Relative entropy and additive white Gaussian noise channels.

The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels

Learning Outcome

After completion of the module the student is able to understand, apply, analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.


Basic principles of probability theory and statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

VU 4 Information Theory

and singular or moved dates

Learning and Teaching Methods

Lerning method:
Lectures, tutorials, excercises, individual study.

Teaching method:
The students are instructed in an explorative teaching style.


- Lecture notes
- Problem sheets
- Presentation slides


Further reading:
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition

Module Exam

Description of exams and course work

During a written exam (90 min) studets proof their ability to apply principles of information theory by answering questions and describing a codes solution as well as a mathematic description for a given problem.

Exam Repetition

There is a possibility to take the exam in the following semester.

Top of page