Introduction to Scientific Machine Learning for Engineers

This course presents the fundamentals of machine learning building on a basic understanding of linear algebra and the axiomatic description of probability theory. Starting with supervised learning basic regression approaches are being discussed, culminating in generalized linear models. Starting with support vector machines various kernel approaches such as Gaussian processes are then covered. We subsequently move on to the general class of neural network methods, their training via backpropagation, bias vs. variance trade-offs, regularization and modern classes of neural networks. The classes covered in this course are recurrent neural networks, convolutional neural networks, generative adversarial networks, and the more modern transformer networks. The course subsequently culminates in an introduction to variational inference, autoencoders and principal component analysis. The content is subject to change based on progress during the semester.

After successfully completing the module, students are able to
- understand Kernel Methods.
- comprehend the construct of neural networks and training via backpropagation.
- understand which neural network class is best suited for a specific application.
- develop the ability to construct a machine learning workflow from basic building blocks.

- Linear Algebra
- Basics of Probability Theory
- Foundational Understanding of Programming

The module consist of a lecture, during which the class material is communicated by oral presentation with the aid of powerpoint presentations and state-of-the-art software tools to develop an understanding of the implementation in practice. Representative problems and code exercises eludicate the essential concepts of machine learning, such as kernel methods, and neural networks. Background reading recommendation are given during class. In addition to the online exercises students can discuss their problems with the teaching personnel in order to deepen the knowledge and explore further applications of the methods.
Through instruction and self-learning the students learn the construction behind neural networks, the up- and downsides of training via backpropagation and a problem-specific grasp for the suitability of neural network classes. Thus, their foundational knowledge is gained to subsequently construct a machine learning workflow from basic building blocks.

Powerpoint slides, online exercises, and source code.

Lecture notes with references are available for download. Autograded computer exercises are available as preparation for the final project. The main references are:
- Pattern Recognition and Machine Learning, by Christopher M. Bishop
- Machine Learning: A Probabilistic Perspective, by Kevin P. Murphy

The module examination takes place in the form of a written exam (pen and paper examination, no auxiliary tools allowed) of 60 minutes. With a mix of comprehension questions and simple mathematical calculations the ability of the student to demonstrate an understanding of kernel methods, neural networks and their training is assessed.