Advanced Deep Learning for Computer Vision: Visual Computing
This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.
IN2390 is a semester module in English language at Master’s level which is offered every semester.
This Module is included in the following catalogues within the study programs in physics.
- Catalogue of non-physics elective courses
|Total workload||Contact hours||Credits (ECTS)|
|240 h||75 h||8 CP|
Content, Learning Outcome and Preconditions
Common lectures with “ADL4CV: Dynamic Vision” lecture:
- Recap of Neural Networks and CNNs
- Advanced Auto-encoders: Probabilistic approaches and the mathematical foundations (e.g., variational auto-encoders)
- Generative Adversarial Networks (from Goodfellow to CycleGANs and Progressive GANs)
- Autoregressive Networks and their parallelization
- Probabilistic vs deterministic generative methods
- Graph neural networks
- Transformer Networks
- Open Problems in Deep Learning for Computer Vision
Unique lectures for this module:
- Multi-dimensional CNNs: from audio to 3D scene environments; 3D vs multi-view CNNs, sparse CNNs (e.g., Octrees);
- Pointer Networks (focus on Scene Understanding, Meshes and 3D Geometry)
- Neural rendering: From Novel View Point synthesis, Video Generation and Editing
- Deep Fakes: Creation and Detection
- CNNs on meshes: learning on structured and unstructured graphs. Mesh data structures and derived convolution operators on meshes using differential geometry.
MA0901 Linear Algebra for Informatics
IN2346 Introduction to Deep Learning (expert knowledge required!)
This is the advanced lecture for deep learning with a specific focus on computer vision. Taking the “Introduction to Deep Learning” course is expected.
Courses, Learning and Teaching Methods and Literature
Courses and Schedule
|VO||2||Advanced Deep Learning for Computer Vision: Visual Computing (IN2390)||Chen, Z. Franzmann, A. Nie, Y. Nießner, M. Rössle, B. … (insgesamt 6)||
Wed, 10:00–12:00, MI 01.09.014
Learning and Teaching Methods
The practical sessions will be key, students shall get familiar with Deep Learning through hours of training and testing. They will work with PyTorch and implement advanced network architectures. The project will have a focus on visual computing, including the following topics:
- neural rendering
- generative neural networks (GANs)
- neural radiance fields
- deep fake generation
- media forensics (forgery detection)
- scene reconstruction (multi-view, depth sensors, etc.)
- generative geometric models
- semantic scene understanding (object detection, instance segmentation, semantic segmentation)
- 3D scene understanding for autonomous driving (e.g., with Lidar/Radar)
- reinforcement learning (e.g., for 3d modeling, 3d auto-scanning, 3d navigation)
- natural language processing (NLP) for scene understanding
We recommend to take a look at the recent list of publications at https://niessnerlab.org/ to get a better idea of recent research projects.
Description of exams and course work
- The lecture will have reading assignments (e.g., from the DeepLearning book and recent CVPR/ICCV/ECCV papers)
- After each practical session, the students will have to provide the written working code to the teaching assistant for evaluation. The students will be awarded a bonus in case they successfully complete all practical assignments.
- In the written exam, we will ask questions regarding lecture theory
- In addition, to the written exam, the results of the projects will be evaluated; we will evaluate projects on a (bi-)weekly basis including reports (33.33%), oral presentations (33.33%), and code/submissions (33.33%).
The exam may be repeated at the end of the semester.