Introduction to Deep Learning

- Introduction to the history of Deep Learning and its applications.
- Machine learning basics 1: linear classification, maximum likelihood
- Machine learning basics 2: logistic regression, perceptron
- Introduction to neural networks and their optimization
- Stochastic Gradient Descent (SGD) and Back-propagation
- Training Neural Networks Part 1:
regularization, activation functions, weight initialization, gradient flow, batch normalization, hyperparameter optimization
- Training Neural Networks Part 2: parameter updates, ensembles, dropout
- Convolutional Neural Networks, ConvLayers, Pooling, etc.
- Applications of CNNs: e.g., object detection (from MNIST to ImageNet), visualizing CNN (DeepDream)
- Overview and introduction to Recurrent networks and LSTMs
- Recent developments in deep learning in the community
- Overview of research and introduction to advanced deep learning lectures.

Upon completion of this module, students will have acquired theoretical concepts behind neural networks, and in particular Convolutional Neural Networks, as well as experience on solving practical real-world problems with deep learning. They will be able to solve tasks such as digit recognition or image classification.

Programming knowledge is expected. At least one programming language should be known, preferably Python.

MA0902 Analysis for Informatics
MA0901 Linear Algebra for Informatics

The lectures will provide extensive theoretical aspects of neural networks and in particular deep learning architectures; e.g., used in the field of Computer Vision.
The practical sessions will be key, students shall get familiar with Deep Learning through hours of training and testing. They will get familiar with frameworks like PyTorch, so that by the end of the course they are capable of solving practical real-world problems with Deep Learning.

Projector, blackboard, PC

- Slides given during the course
- www.deeplearningbook.org

- Written test of 90 minutes at the end of the course.
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation. The students will be awarded a bonus in case they successfully complete all practical assignments.

The exam takes the form of a written test. Questions allow to assess acquaintance with the basic concepts and algorithms of deep learning concepts, in particular how to train neural networks. Students demonstrate the ability to design, train, and optimize neural network architectures, and how to apply the learning frameworks to real-world problems (e.g., in computer vision). An important aspect for the student is to understand the basic theory behind the training process, which is mainly coupled with optimization strategies involving backprop and SGD. Students can use networks in order to solve classification and regression tasks (partly motivated by visual data).