Advanced Deep Learning for Robotics

This is the advanced deep learning lecture with a specific focus on Robotics and deep reinforcement learning (including a guest lecture from DeepMind). Taking the “Introduction to Deep Learning” course is expected.

1. Introduction & Recap of Deep Learning
2. Advanced Network Architectures & Recursive Neural Networks (LSTMs)
3. Hyperparameter & Architecture Search
Bayesian optimization, network architecture search (NAS)
4. Adversarial Samples & Adversarial Training
5. Bayesian Deep Learning
Bayesian learning, variational inference, Monte Carlo dropout method
6. Generative Models: VAEs & GANs
variational auto-encoders, generative neural networks (WGAN-GP)
7. Data Efficient Learning: Transfer & Semi-Supervised Learning
8. Deep Reinforcement Learning I
MDPs, Q-iteration, Bellman equation, deep Q-learning, example: Atari-games
9. Deep Reinforcement Learning II
policy gradients, REINFORCE, actor-critic algorithm, TRPO, PPO, robotic applications
10. Deep Reinforcement Learning III
advanced methods: DDPG, soft Q-learning, soft actor-critic (SAC), robotic applications
11. Deep Reinforcement Learning IV
model-based DRL, MCTS + learned heuristics, AlphaZero, model learning, PDDM, robotics
applications
12. Guest Lecture from DeepMind
Recent Developments in Deep Reinforcement Learning for Robotics

Upon completion of this module, students will have acquired extensive theoretical concepts behind advanced architectures of neural networks and state of the art deep reinforcement learning methods in the context of robotic tasks. In addition to the theoretical foundations, a significant aspect lies on the practical realization of deep reinforcement learning (DRL) methods in robotic scenarios.

MA0902 Analysis for Informatics
MA0901 Linear Algebra for Informatics

IN2346 Introduction to Deep Learning (expert knowledge required!)

The lectures will provide extensive theoretical aspects of advanced deep learning architectures and specifically deep reinforcement learning methods in the field of robotics. The lecture will have reading assignments (e.g., from the DeepLearning book and recent RSS/ICRL/ICRA/IROS papers).
The practical sessions will be key, students shall get familiar with esp. Deep Reinforcement Learning through hours of training and testing. The students will do a semester-long project in teams of 2 with weekly presentations and tutoring of the projects throughout the semester. They will work with TensorFlow and OpenAI Gym and implement advanced deep reinforcement learning methods for state of the art robotic problems. For each student, $1000 credits in the Google Cloud are available via a Google Educational Grant.

Projector, blackboard, PC

- Slides given during the course
- www.deeplearningbook.org

- Written test of 60 minutes at the end of the course (for lecture)
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation.
- In the written exam (50% of the final grade), we will ask questions regarding lecture theory
- In addition, to the written exam, the results of the projects will be evaluated (50% of the final grade); we will evaluate projects on a (bi-) weekly basis including reports (33.33%), oral presentations (33.33%), and code/submissions (33.33%).