Mathematical Foundations of Machine Learning
Module MA4801
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
Module version of WS 2020/1 (current)
There are historic module descriptions of this module. A module description is valid until replaced by a newer one.
Whether the module’s courses are offered during a specific semester is listed in the section Courses, Learning and Teaching Methods and Literature below.
| available module versions | ||
|---|---|---|
| WS 2020/1 | SS 2020 | WS 2015/6 |
Basic Information
MA4801 is a semester module in English language at Master’s level which is offered in summer semester.
This Module is included in the following catalogues within the study programs in physics.
- Specialization Modules in Elite-Master Program Theoretical and Mathematical Physics (TMP)
| Total workload | Contact hours | Credits (ECTS) |
|---|---|---|
| 180 h | 60 h | 6 CP |
Content, Learning Outcome and Preconditions
Content
A Neural Networks
(1) the perceptron
(2) network architecture (feedforward networks)
(3) Kolmogorov superposition theorem
(4) backpropagation and learning algorithms
(5) approximation properties of different architectures
B Kernel Methods
(1) positive definite kernels
(2) Mercer kernels
(3) reproducing kernel Hilbert spaces
(4) regularization techniques and support vector machines
(5) representer theorem for the minimizer
(6) numerical algorithms for SVM's
C Qualitative Theory
(1) loss functions
(2) risk functionals
(3) empirical risk minimization
(4) bias-variance dilemma
(5) consitency
(6) complexity bounds
(1) the perceptron
(2) network architecture (feedforward networks)
(3) Kolmogorov superposition theorem
(4) backpropagation and learning algorithms
(5) approximation properties of different architectures
B Kernel Methods
(1) positive definite kernels
(2) Mercer kernels
(3) reproducing kernel Hilbert spaces
(4) regularization techniques and support vector machines
(5) representer theorem for the minimizer
(6) numerical algorithms for SVM's
C Qualitative Theory
(1) loss functions
(2) risk functionals
(3) empirical risk minimization
(4) bias-variance dilemma
(5) consitency
(6) complexity bounds
Learning Outcome
After successful completion of the module students are able to understand and apply the basic notions, concepts, and methods of mathine learning. They are able to construct and implement a neural network and to discuss ist approximation properties. They understood the theory of kernel methods in reproducing kernel Hilbert spaces, and they know how to apply it to provide nonlinear regression of data. They know how to assess the statistical efficiency of a machine learning method.
Preconditions
MA1001 Analysis 1, MA1002 Analysis 2, MA1101 Linear Algebra 1, MA1102 Linear Algebra 2, MA1401 Introduction to Probability Theory, MA2003 Measure and Integration, MA3001 Funktionalanalysis. Suggested optional: MA2501 Algorithmic Discrete Mathematics, MA2503 Introduction to Nonlinear Optimization
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
| Type | SWS | Title | Lecturer(s) | Dates | Links |
|---|---|---|---|---|---|
| VO | 2 | Mathematical Foundations of Machine Learning | Caro, M. Wolf, M. |
Mon, 10:15–11:45, virtuell |
eLearning documents |
| UE | 2 | Mathematical Foundations of Machine Learning (Exercise Session) | Caro, M. Wolf, M. |
documents |
Learning and Teaching Methods
lecture, exercise module
Media
The following media are used:
- Blackboard
- Slides
- Blackboard
- Slides
Literature
C.M. Bishop, Pattern Recognition and Machine Learning, Springer 2006.
D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge Univ. Press 2003.
V.N. Vapnik, Statistical Learning Theory, Wiley 1998.
T. Hastie, R. Tibshirani, J. Fiedman, The Elements of Statistical Learning Theory, Springer 2009.
D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge Univ. Press 2003.
V.N. Vapnik, Statistical Learning Theory, Wiley 1998.
T. Hastie, R. Tibshirani, J. Fiedman, The Elements of Statistical Learning Theory, Springer 2009.
Module Exam
Description of exams and course work
The exam will be in written form (60 minutes). Students demonstrate that they have gained deeper knowledge of definitions and main tools and results of machine learning. The students are expected to be able to derive the methods, to explain their properties, and to apply them to specific examples.
Exam Repetition
The exam may be repeated at the end of the semester.