Advanced Deep Learning for Computer Vision: Dynamic Vision
Module IN2389
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
Basic Information
IN2389 is a semester module in English language at Master’s level which is offered every semester.
This Module is included in the following catalogues within the study programs in physics.
- Catalogue of non-physics elective courses
Total workload | Contact hours | Credits (ECTS) |
---|---|---|
240 h | 75 h | 8 CP |
Content, Learning Outcome and Preconditions
Content
Note, this lecture is closely related (and mutually exclusive) to the lecture “Advanced Deep Learning for Computer Vision: Visual Computing”. The two lectures share some theoretical content, but the “Dynamic Vision” module provides a clear focus on video analysis tasks, which is especially important for the practical part in the form of a semester-long project.
Common lectures with “ADL4CV: Visual Computing” lecture:
- Recap of Neural Networks and CNNs
- Advanced Auto-encoders: Probabilistic approaches and the mathematical foundations (e.g., variational auto-encoders)
- Generative Adversarial Networks (from Goodfellow to CycleGANs and Progressive GANs)
- Autoregressive Networks and their parallelization
- Probabilistic vs deterministic generative methods
- Graph neural networks
- Transformers
- Open Problems in Deep Learning for Computer Vision
Unique lectures for this module:
- Siamese neural networks and deep metric learning
- Visualization techniques
- Multi-dimensional CNNs: spatio-temporal neural networks.
- Active learning and semi-supervised learning
Common lectures with “ADL4CV: Visual Computing” lecture:
- Recap of Neural Networks and CNNs
- Advanced Auto-encoders: Probabilistic approaches and the mathematical foundations (e.g., variational auto-encoders)
- Generative Adversarial Networks (from Goodfellow to CycleGANs and Progressive GANs)
- Autoregressive Networks and their parallelization
- Probabilistic vs deterministic generative methods
- Graph neural networks
- Transformers
- Open Problems in Deep Learning for Computer Vision
Unique lectures for this module:
- Siamese neural networks and deep metric learning
- Visualization techniques
- Multi-dimensional CNNs: spatio-temporal neural networks.
- Active learning and semi-supervised learning
Learning Outcome
Upon completion of this module, students will have acquired extensive theoretical concepts behind advanced architectures of neural networks, in particular in the context of video analysis for computer vision tasks. In addition to the theoretical foundations, a significant aspect lies on the practical realization and training of neural networks.
Preconditions
MA0902 Analysis for Informatics
MA0901 Linear Algebra for Informatics
IN2346 Introduction to Deep Learning (expert knowledge required!)
IN2375 Computer Vision 3: Detection, Segmentation, and Tracking
This is the advanced lecture for deep learning with a specific focus on computer vision for video analysis. Taking the “Introduction to Deep Learning” course is expected.
MA0901 Linear Algebra for Informatics
IN2346 Introduction to Deep Learning (expert knowledge required!)
IN2375 Computer Vision 3: Detection, Segmentation, and Tracking
This is the advanced lecture for deep learning with a specific focus on computer vision for video analysis. Taking the “Introduction to Deep Learning” course is expected.
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
Type | SWS | Title | Lecturer(s) | Dates | Links |
---|---|---|---|---|---|
VO | 2 | Advanced Deep Learning for Computer Vision: Dynamic Vision (IN2389) | Dendorfer, P. Elezi, I. Leal-Taixe, L. |
Mon, 10:00–12:00, virtuell |
|
PR | 3 | Practical Course - Advanced Deep Learning for Computer Vision: Dynamic Vision (IN2389) |
Dendorfer, P.
Elezi, I.
Toker, A.
Responsible/Coordination: Leal-Taixe, L. |
Wed, 14:00–16:00, MI 02.09.023 |
Learning and Teaching Methods
The lectures will provide extensive theoretical aspects of neural networks and in particular deep learning architectures, specifically for advanced methods in the field of Computer Vision for video analysis.
The practical sessions will be key, students shall get familiar with Deep Learning through hours of training and testing. They will work with PyTorch and implement advanced network architectures. The project will have a clear focus on video analysis, following the research topics of the group, including the following topics:
- Multi-object tracking
- Video object segmentation
- Trajectory prediction
- Panoptic segmentation
- Active learning for videos
- Video anonymization with GANs
- Pose estimation
We recommend to take a look at the recent list of publications at https://dvl.in.tum.de to get a better idea of recent research projects.
The practical sessions will be key, students shall get familiar with Deep Learning through hours of training and testing. They will work with PyTorch and implement advanced network architectures. The project will have a clear focus on video analysis, following the research topics of the group, including the following topics:
- Multi-object tracking
- Video object segmentation
- Trajectory prediction
- Panoptic segmentation
- Active learning for videos
- Video anonymization with GANs
- Pose estimation
We recommend to take a look at the recent list of publications at https://dvl.in.tum.de to get a better idea of recent research projects.
Media
Projector, blackboard, PC
Literature
- Slides given during the course
- www.deeplearningbook.org
- www.deeplearningbook.org
Module Exam
Description of exams and course work
- Written test of 60 minutes at the end of the course (for lecture)
- The lecture will have reading assignments (e.g., from the DeepLearning book and recent CVPR/ICCV/ECCV papers)
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation. The students will be awarded a bonus in case they successfully complete all practical assignments.
- In the written exam, we will ask questions regarding lecture theory
- In addition, to the written exam, the results of the projects will be evaluated; we will evaluate projects on a (bi-)weekly basis including reports (33.33%), oral presentations (33.33%), and code/submissions (33.33%).
- The lecture will have reading assignments (e.g., from the DeepLearning book and recent CVPR/ICCV/ECCV papers)
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation. The students will be awarded a bonus in case they successfully complete all practical assignments.
- In the written exam, we will ask questions regarding lecture theory
- In addition, to the written exam, the results of the projects will be evaluated; we will evaluate projects on a (bi-)weekly basis including reports (33.33%), oral presentations (33.33%), and code/submissions (33.33%).
Exam Repetition
The exam may be repeated at the end of the semester.