Information Theory

Module EI7431

This Module is offered by Chair of Communications Engineering (Prof. Kramer).

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

EI7431 is a semester module in English language at Master’s level which is offered in winter semester.

This Module is included in the following catalogues within the study programs in physics.

  • Catalogue of non-physics elective courses
Total workloadContact hoursCredits (ECTS)
150 h 60 h 5 CP

Content, Learning Outcome and Preconditions


Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity. Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-Willems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channnels, Capacity. Relative entropy and additive white Gaussian noise channels. The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels

Learning Outcome

After completion of the module the student is able to understand, apply, analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.


Basic principles of probability theory and statistics

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

VU 4 Information Theory Dienstag, 08:00–09:30
Dienstag, 15:00–16:30
sowie einzelne oder verschobene Termine

Learning and Teaching Methods

Lerning method: Lectures, tutorials, excercises, individual study. Teaching method: The students are instructed in an explorative teaching style.


- Lecture notes - Problem sheets - Presentation slides


Further reading: - Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition

Module Exam

Description of exams and course work

During a written exam (90 min) studets proof their ability to apply principles of information theory by answering questions and describing a codes solution as well as a mathematic description for a given problem.

Exam Repetition

There is a possibility to take the exam in the following semester.

Condensed Matter

When atoms interact things can get interesting. Fundamental research on the underlying properties of materials and nanostructures and exploration of the potential they provide for applications.

Nuclei, Particles, Astrophysics

A journey of discovery to understanding our world at the subatomic scale, from the nuclei inside atoms down to the most elementary building blocks of matter. Are you ready for the adventure?


Biological systems, from proteins to living cells and organisms, obey physical principles. Our research groups in biophysics shape one of Germany's largest scientific clusters in this area.