Mathematical Foundations of Machine Learning

Module MA4801

This Module is offered by TUM Department of Mathematics.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Module version of WS 2020/1 (current)

There are historic module descriptions of this module. A module description is valid until replaced by a newer one.

Whether the module’s courses are offered during a specific semester is listed in the section Courses, Learning and Teaching Methods and Literature below.

available module versions
WS 2020/1	SS 2020	WS 2015/6

Basic Information

MA4801 is a semester module in English language at Master’s level which is offered in summer semester.

This Module is included in the following catalogues within the study programs in physics.

Specialization Modules in Elite-Master Program Theoretical and Mathematical Physics (TMP)

Total workload	Contact hours	Credits (ECTS)
180 h	60 h	6 CP

Content, Learning Outcome and Preconditions

Content

A Neural Networks
(1) the perceptron
(2) network architecture (feedforward networks)
(3) Kolmogorov superposition theorem
(4) backpropagation and learning algorithms
(5) approximation properties of different architectures
B Kernel Methods
(1) positive definite kernels
(2) Mercer kernels
(3) reproducing kernel Hilbert spaces
(4) regularization techniques and support vector machines
(5) representer theorem for the minimizer
(6) numerical algorithms for SVM's
C Qualitative Theory
(1) loss functions
(2) risk functionals
(3) empirical risk minimization
(4) bias-variance dilemma
(5) consitency
(6) complexity bounds

Learning Outcome

After successful completion of the module students are able to understand and apply the basic notions, concepts, and methods of mathine learning. They are able to construct and implement a neural network and to discuss ist approximation properties. They understood the theory of kernel methods in reproducing kernel Hilbert spaces, and they know how to apply it to provide nonlinear regression of data. They know how to assess the statistical efficiency of a machine learning method.

Preconditions

MA1001 Analysis 1, MA1002 Analysis 2, MA1101 Linear Algebra 1, MA1102 Linear Algebra 2, MA1401 Introduction to Probability Theory, MA2003 Measure and Integration, MA3001 Funktionalanalysis. Suggested optional: MA2501 Algorithmic Discrete Mathematics, MA2503 Introduction to Nonlinear Optimization

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

SS 2021 SS 2020 SS 2018 SS 2016

Type	SWS	Title	Lecturer(s)	Dates	Links
VO	2	Mathematical Foundations of Machine Learning	Caro, M. Wolf, M.	Mon, 10:15–11:45, virtuell	eLearning documents
UE	2	Mathematical Foundations of Machine Learning (Exercise Session)	Caro, M. Wolf, M.		documents

Learning and Teaching Methods

lecture, exercise module

Media

The following media are used:
- Blackboard
- Slides

Literature

C.M. Bishop, Pattern Recognition and Machine Learning, Springer 2006.
D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge Univ. Press 2003.
V.N. Vapnik, Statistical Learning Theory, Wiley 1998.
T. Hastie, R. Tibshirani, J. Fiedman, The Elements of Statistical Learning Theory, Springer 2009.

Module Exam

Description of exams and course work

The exam will be in written form (60 minutes). Students demonstrate that they have gained deeper knowledge of definitions and main tools and results of machine learning. The students are expected to be able to derive the methods, to explain their properties, and to apply them to specific examples.

Exam Repetition

The exam may be repeated at the end of the semester.

Current exam dates

Currently TUMonline lists the following exam dates. In addition to the general information above please refer to the current information given during the course.

Title
Time	Location	Info	Registration
Mathematical Foundations of Machine Learning
	2501