de | en

Advanced Deep Learning for Robotics

Module IN2349

This Module is offered by TUM Department of Informatics.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

IN2349 is a semester module in English language at Master’s level which is offered every semester.

This Module is included in the following catalogues within the study programs in physics.

  • Focus Area Bio-Sensors in M.Sc. Biomedical Engineering and Medical Physics
Total workloadContact hoursCredits (ECTS)
180 h 60 h 6 CP

Content, Learning Outcome and Preconditions

Content

This is the advanced deep learning lecture with a specific focus on Robotics and deep reinforcement learning (including a guest lecture from DeepMind). Taking the “Introduction to Deep Learning” course is expected.

1. Introduction & Recap of Deep Learning
2. Advanced Network Architectures & Recursive Neural Networks (LSTMs)
3. Hyperparameter & Architecture Search
Bayesian optimization, network architecture search (NAS)
4. Adversarial Samples & Adversarial Training
5. Bayesian Deep Learning
Bayesian learning, variational inference, Monte Carlo dropout method
6. Generative Models: VAEs & GANs
variational auto-encoders, generative neural networks (WGAN-GP)
7. Data Efficient Learning: Transfer & Semi-Supervised Learning
8. Deep Reinforcement Learning I
MDPs, Q-iteration, Bellman equation, deep Q-learning, example: Atari-games
9. Deep Reinforcement Learning II
policy gradients, REINFORCE, actor-critic algorithm, TRPO, PPO, robotic applications
10. Deep Reinforcement Learning III
advanced methods: DDPG, soft Q-learning, soft actor-critic (SAC), robotic applications
11. Deep Reinforcement Learning IV
model-based DRL, MCTS + learned heuristics, AlphaZero, model learning, PDDM, robotics
applications
12. Guest Lecture from DeepMind
Recent Developments in Deep Reinforcement Learning for Robotics

Learning Outcome

Upon completion of this module, students will have acquired extensive theoretical concepts behind advanced architectures of neural networks and state of the art deep reinforcement learning methods in the context of robotic tasks. In addition to the theoretical foundations, a significant aspect lies on the practical realization of deep reinforcement learning (DRL) methods in robotic scenarios.

Preconditions

MA0902 Analysis for Informatics
MA0901 Linear Algebra for Informatics

IN2346 Introduction to Deep Learning (expert knowledge required!)

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

TypeSWSTitleLecturer(s)DatesLinks
VI 4 Advanced Deep Learning for Robotics (IN2349) Bäuml, B. Thu, 12:00–15:30, virtuell
Thu, 12:00–15:30, virtuell
eLearning
documents

Learning and Teaching Methods

The lectures will provide extensive theoretical aspects of advanced deep learning architectures and specifically deep reinforcement learning methods in the field of robotics. The lecture will have reading assignments (e.g., from the DeepLearning book and recent RSS/ICRL/ICRA/IROS papers).
The practical sessions will be key, students shall get familiar with esp. Deep Reinforcement Learning through hours of training and testing. The students will do a semester-long project in teams of 2 with weekly presentations and tutoring of the projects throughout the semester. They will work with TensorFlow and OpenAI Gym and implement advanced deep reinforcement learning methods for state of the art robotic problems. For each student, $1000 credits in the Google Cloud are available via a Google Educational Grant.

Media

Projector, blackboard, PC

Literature

- Slides given during the course
- www.deeplearningbook.org

Module Exam

Description of exams and course work

- Written test of 60 minutes at the end of the course (for lecture)
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation.
- In the written exam (50% of the final grade), we will ask questions regarding lecture theory
- In addition, to the written exam, the results of the projects will be evaluated (50% of the final grade); we will evaluate projects on a (bi-) weekly basis including reports (33.33%), oral presentations (33.33%), and code/submissions (33.33%).

Exam Repetition

There is a possibility to take the exam in the following semester.

Top of page