Information Theory

Module EI5005

This Module is offered by TUM Department of Electrical and Computer Engineering.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

EI5005 is a semester module in English language at Master’s level which is offered in winter semester.

This module description is valid to SS 2013.

Total workload	Contact hours	Credits (ECTS)
90 h	45 h	3 CP

Content, Learning Outcome and Preconditions

Content

Review of probability theory. Uncertainty and mutual information.
Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Arithmetic codes, Lempel-Ziv codes, Elias-Willems codes.
Channel coding: Discrete Memoryless Channnels, Capacity, Fano's inequality, Random Coding. Additive white Gaussian noise channels.

Learning Outcome

The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.

Preconditions

Basic principles of probability theory and statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

WS 2013/4 WS 2012/3

Type	SWS	Title	Lecturer(s)	Dates	Links
VU	3	Information Theory and Source Coding		Tue, 08:45–09:30 Tue, 15:00–16:30	eLearning documents

Learning and Teaching Methods

Lerning method:
Lectures, tutorials, excercises, individual study.

Teaching method:
The students are instructed in an explorative teaching style.

Media

- Lecture notes
- Problem sheets
- Presentation slides

Literature

Further reading:
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition