Applied Machine Learning (Deep Learning for Multimedia
and
Practical Concepts of Machine Learning)

1. Deep Learning for Multimedia:
Content generated for human consumption in the form of video, text, or audio, is unstructured from a machine perspective since the contained information is not readily available for processing. Information extraction from unstructured data describes therefore how one can extract the salient information from generic content in order to generate a descriptive structured representation. The thus created meta-data can then be further processed automatically, in particular for creating models explaining or predicting samples e.g. in recommendation systems.
Aim of this lecture is therefore to introduce the methods, algorithms and underlying machine learning concepts for extracting information from audio, visual, and textual unstructured content using state-of-the art algorithms, especially deep learning based algorithms and architectures e.g. CNN, Autoencoder, LTSM. In addition, existing frameworks and libraries (e.g. Keras, Scikit-learn) and how to use them with audio, visual, and textual content countered in (multi-) media applications and services will be discussed.
The following topics will be covered:
- Why information extraction?
- Introduction to deep learning
- Image/video content
- Object recognition
- Face recognition
- Character recognition (OCR)
- Quality of Experience (QoE)
- Audio/textual content
- Automatics speech recognition (ASR)
- Natural language processing (NLP)
- Python eco-system of frameworks/libraries for information extraction
Selected topics will be examined more in-depth during the lecture and the team oriented semester project.

2. Practical Concepts of Machine Learning:
The course Practical Concepts of Machine Learning focuses on the acquiring practical skills for applying concepts of machine learning in analyzing data, which come from a wide range of data sources.
We will discuss and exercise methods for
▪ planning a data collection campaign, a test procedure or measurements and experiments
▪ exploring the collected data to search for structure and meaningful patterns hidden in the data
▪ building prediction models and classifiers to capture the essence of the phenomena comprised in data
▪ exploiting human cognition and integrating domain knowledge

All these methods are presented along practical examples of data processing and analyzing, covering a wide range of applications, which are representative to the field of computer engineering. The style of the course is focusing on practical aspects built on top of theoretical foundations. The presented methods directly will lead to Data Mining and Big Data topics.
We will implement numerical algorithms, visualize and process the data, evaluate and validate prediction models and discuss various implementation platforms (computer architectures) for efficient data analysis.

After the successful participation in the module, students know the methods, algorithms and underlying machine learning concepts for extracting information from audio, visual, and textual unstructured content. They understand the real-life constraints and resulting requirements for the design, implementation, and application of information extraction from unstructured data. Students are able to apply and modify existing information extraction algorithms, taking into account real-life requirements. They are able to evaluate information extraction algorithms and methodologies with respect to their suitability for specific applications or services.

The lecture assumes basic knowledge of general topics discussed at undergraduate level (BSc.) in one of the following areas:
- Computer engineering
- Communications engineering
- Multimedia technology and human machine interaction

Additionally, basic knowledge of Python (or the motivation to learn it) is recommended.

The course consists of frontal teaching and discussions about current research questions using literature.

The written assignments (homework) and the project work enable the students to apply the learned concepts from the lecture to real-life applications and problems. During the project, the students will be supported by dedicated tutorials.

The following media will be used
- Example code for algorithms
- Slides
- Lecture notes
- Videos

“Deep learning,” I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, MIT press, Cambridge, 2016

“Automatic Speech Recognition: A Deep Learning Approach,” D. Yu, L. Deng, Springer, London, 2015

“Design of Video Quality Metrics with Multi-Way Data Analysis: A data driven approach, “ C. Keimel, Springer Singapore, 2016

“Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems,” A. Géron, O’Reilly Media, 2017

The assessment of the students‘ participation in the lecture is split into three components:

• Theoretical knowledge about information extraction from unstructured data will be evaluated in a 30 minute oral final exam (closed book)
• The individual students’ abilities to solve problems in the area of information extraction from unstructured data by applying machine learning and the thus necessary ability to apply and adapt the theoretical knowledge will be assed in written assignments (homework) during the semester. The assignments consist of writing short papers discussing specific problems in the discussed topics.
• The ability of students to use the concepts of information extracting using machine learning in real-life applications taking into account constraints in realistic use-cases will be assessed by the results of the project and a corresponding presentation of these results. Moreover, the students’ general abilities of successful performing in a team including self-organisation will also be assessed by the successful completion of the project.

The overall grade is the weighted sum of the above components:
- 30 % Final exam
- 20 % Homework (2 graded assignments)
- 50 % Project