de | en

Solving Inverse Problems with Deep Learning (Deep learning and inverse problems)

Module EI71068

This Module is offered by TUM Department of Electrical and Computer Engineering.

This module handbook serves to describe contents, learning outcome, methods and examination type as well as linking to current dates for courses and module examination in the respective sections.

Basic Information

EI71068 is a semester module in English language at which is offered in summer semester.

This Module is included in the following catalogues within the study programs in physics.

  • Catalogue of non-physics elective courses
Total workloadContact hoursCredits (ECTS)
180 h 60 h 6 CP

Content, Learning Outcome and Preconditions

Content

There is a long history of algorithmic development for solving inverse problems arising in sensing and imaging systems and beyond. Examples include medical and computational imaging, compressive sensing, as well as community detection in networks. Until recently, most algorithms for solving inverse problems in the imaging and beyond were based on static signal models derived from physics or intuition, such as wavelets or sparse representations.

Today, the best performing approaches for the aforementioned image reconstruction and sensing problems are based on deep learning, which learn various elements of the method including i) signal representations, ii) stepsizes and parameters of iterative algorithms, iii) regularizers, and iv) entire inverse functions. Motivated by those success stories, researchers are redesigning traditional imaging and sensing systems, and deep learning based signal reconstruction methods are starting to be used in important imaging technologies, for example in GEs newest computational tomography scanners and in the newest generation of the iPhone.

This course gives a graduate/master level introduction to deep learning based imaging. The course first introduces classical approaches to solving inverse problems and then aims to explain the recent advances of deep neural network based approaches for solving inverse problems in the imaging sciences.

Topics include classical sparse models, optimization for fitting classical methods and for training deep networks, unrolled algorithms, convolutional neural networks for image recovery and generation, generative models for image recovery, un-trained neural networks for signal recovery. The course ends with a brief outlook on how to apply those methods beyond image recovery for the recovery of a variety of other signals.

Learning Outcome

Upon successful completion of the module, students will be able to i) apply learning based methods to inverse problems such as signal and image recovery from few and noisy images, ii) evaluate their theoretical foundations and limits, and iii) critically evaluate papers and methods in that area, and iv) design variations of existing methods.

Preconditions

Analysis, an introduction to probability and statistics, and linear algebra. An introduction to machine learning is very helpful, but not necessary.

Courses, Learning and Teaching Methods and Literature

Courses and Schedule

TypeSWSTitleLecturer(s)DatesLinks
VO 4 Deep learning and inverse problems Heckel, R. Klug, T. Mansour, Y. Tue, 13:15–16:30, N1080ZG
and singular or moved dates

Learning and Teaching Methods

The course will take place in two sessions on the same day: In the first session the foundations of a topic are explained during lectures, with a focus on methods, ideas, and the theory behind the methods.
In the second session, we will discuss the method in more detail, go through a problem, or discuss a recent paper on that topic. If we discuss a paper, all students will need to read that paper before class so that we can have a meaningful discussion.

Media

The lecture will be given via zoom. Lecture notes and exercises will be distributed.

Literature

Lecture notes will be distributed.

Module Exam

Description of exams and course work

Students take a written exam of two hours duration. The exam consists of questions on the theory and algorithms of signal recovery with deep neural networks. The exam tests whether students can analyse, evaluate, and design solvers for inverse problems with deep networks. The exam is open-book, thus lecture notes and any other notes are permitted, but no computational devices are needed or are allowed.

Besides the written exam, 20% of the grade will be either through a presentation of a paper or through homework submission. In either case we will test whether students can analyse, evaluate, and design solvers for inverse problems with deep networks.
Specifically, if sufficiently few students are enrolled, each student has to present or defend a paper in the discussion session, and the evaluation of the paper, as well as the quality of presenting the arguments will count 20% towards the final grade.
If more students are enrolled than enabling each student to present a paper, then students will not present papers. In this case the submission of the homeworks which will contain evaluation and design of solvers for inverse problems will count 20% towards the final grade. The homework will contain analysis, evaluation, and design problems on solving inverse problems with deep networks.
The mode will be determined before the first lecture, and be communicated during the first lecture.

Exam Repetition

There is a possibility to take the exam in the following semester.

Current exam dates

Currently TUMonline lists the following exam dates. In addition to the general information above please refer to the current information given during the course.

Title
TimeLocationInfoRegistration
Solving Inverse Problems with Deep Learning
till 2023-01-15 (cancelation of registration till 2023-01-27)
Top of page