Information Theory
Module EI7431
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
Basic Information
EI7431 is a semester module in English language at Master’s level which is offered in winter semester.
This module description is valid from WS 2014/5 to SS 2016.
| Total workload | Contact hours | Credits (ECTS) |
|---|---|---|
| 150 h | 60 h | 5 CP |
Content, Learning Outcome and Preconditions
Content
Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity.
Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-Willems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channnels, Capacity. Relative entropy and additive white Gaussian noise channels.
The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels
Source coding: Discrete Memoryless Sources, Prefix-Free codes, Shannon-Fano codes, Huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-Willems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channnels, Capacity. Relative entropy and additive white Gaussian noise channels.
The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels
Learning Outcome
After completion of the module the student is able to understand, apply, analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.
Preconditions
Basic principles of probability theory and statistics
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
| Type | SWS | Title | Lecturer(s) | Dates | Links |
|---|---|---|---|---|---|
| VU | 4 | Information Theory |
Tue, 08:00–09:30 Tue, 15:00–16:30 and singular or moved dates |
eLearning |
Learning and Teaching Methods
Lerning method:
Lectures, tutorials, excercises, individual study.
Teaching method:
The students are instructed in an explorative teaching style.
Lectures, tutorials, excercises, individual study.
Teaching method:
The students are instructed in an explorative teaching style.
Media
- Lecture notes
- Problem sheets
- Presentation slides
- Problem sheets
- Presentation slides
Literature
Further reading:
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition
Module Exam
Description of exams and course work
During a written exam (90 min) studets proof their ability to apply principles of information theory by answering questions and describing a codes solution as well as a mathematic description for a given problem.
Exam Repetition
There is a possibility to take the exam in the following semester.