Applied Machine Learning (Deep Learning for Multimedia
Practical Concepts of Machine Learning)
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
EI71049 is a semester module in English language at Master’s level which is offered in summer semester.
This Module is included in the following catalogues within the study programs in physics.
- Catalogue of non-physics elective courses
|Total workload||Contact hours||Credits (ECTS)|
|180 h||150 h||6 CP|
Content, Learning Outcome and Preconditions
Content generated for human consumption in the form of video, text, or audio, is unstructured from a machine perspective since the contained information is not readily available for processing. Information extraction from unstructured data describes therefore how one can extract the salient information from generic content in order to generate a descriptive structured representation. The thus created meta-data can then be further processed automatically, in particular for creating models explaining or predicting samples e.g. in recommendation systems.
Aim of this lecture is therefore to introduce the methods, algorithms and underlying machine learning concepts for extracting information from audio, visual, and textual unstructured content using state-of-the art algorithms, especially deep learning based algorithms and architectures e.g. CNN, Autoencoder, LTSM. In addition, existing frameworks and libraries (e.g. Keras, Scikit-learn) and how to use them with audio, visual, and textual content countered in (multi-) media applications and services will be discussed.
The following topics will be covered:
- Why information extraction?
- Introduction to deep learning
- Image/video content
- Object recognition
- Face recognition
- Character recognition (OCR)
- Quality of Experience (QoE)
- Audio/textual content
- Automatics speech recognition (ASR)
- Natural language processing (NLP)
- Python eco-system of frameworks/libraries for information extraction
Selected topics will be examined more in-depth during the lecture and the team oriented semester project.
2. Practical Concepts of Machine Learning:
The course Practical Concepts of Machine Learning focuses on the acquiring practical skills for applying concepts of machine learning in analyzing data, which come from a wide range of data sources.
We will discuss and exercise methods for
▪ planning a data collection campaign, a test procedure or measurements and experiments
▪ exploring the collected data to search for structure and meaningful patterns hidden in the data
▪ building prediction models and classifiers to capture the essence of the phenomena comprised in data
▪ exploiting human cognition and integrating domain knowledge
All these methods are presented along practical examples of data processing and analyzing, covering a wide range of applications, which are representative to the field of computer engineering. The style of the course is focusing on practical aspects built on top of theoretical foundations. The presented methods directly will lead to Data Mining and Big Data topics.
We will implement numerical algorithms, visualize and process the data, evaluate and validate prediction models and discuss various implementation platforms (computer architectures) for efficient data analysis.
- Computer engineering
- Communications engineering
- Multimedia technology and human machine interaction
Additionally, basic knowledge of Python (or the motivation to learn it) is recommended.
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
|VI||3||Applied Machine Learning - Deep Learning for Multimedia||Keimel, C. Volk, T.|
|VI||3||Applied Machine Learning - Practical Concepts of Machine Learning||
Assistants: Paukner, P.
|PR||4||Praktikum Applied Machine Learning||
Assistants: Paukner, P.
Learning and Teaching Methods
The written assignments (homework) and the project work enable the students to apply the learned concepts from the lecture to real-life applications and problems. During the project, the students will be supported by dedicated tutorials.
- Example code for algorithms
- Lecture notes
“Automatic Speech Recognition: A Deep Learning Approach,” D. Yu, L. Deng, Springer, London, 2015
“Design of Video Quality Metrics with Multi-Way Data Analysis: A data driven approach, “ C. Keimel, Springer Singapore, 2016
“Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems,” A. Géron, O’Reilly Media, 2017
Description of exams and course work
• Theoretical knowledge about information extraction from unstructured data will be evaluated in a 30 minute oral final exam (closed book)
• The individual students’ abilities to solve problems in the area of information extraction from unstructured data by applying machine learning and the thus necessary ability to apply and adapt the theoretical knowledge will be assed in written assignments (homework) during the semester. The assignments consist of writing short papers discussing specific problems in the discussed topics.
• The ability of students to use the concepts of information extracting using machine learning in real-life applications taking into account constraints in realistic use-cases will be assessed by the results of the project and a corresponding presentation of these results. Moreover, the students’ general abilities of successful performing in a team including self-organisation will also be assessed by the successful completion of the project.
The overall grade is the weighted sum of the above components:
- 30 % Final exam
- 20 % Homework (2 graded assignments)
- 50 % Project
There is a possibility to take the exam in the following semester.