Solving Inverse Problems with Deep Learning (Deep learning and inverse problems)

There is a long history of algorithmic development for solving inverse problems arising in sensing and imaging systems and beyond. Examples include medical and computational imaging, compressive sensing, as well as community detection in networks. Until recently, most algorithms for solving inverse problems in the imaging and beyond were based on static signal models derived from physics or intuition, such as wavelets or sparse representations.

Today, the best performing approaches for the aforementioned image reconstruction and sensing problems are based on deep learning, which learn various elements of the method including i) signal representations, ii) stepsizes and parameters of iterative algorithms, iii) regularizers, and iv) entire inverse functions. Motivated by those success stories, researchers are redesigning traditional imaging and sensing systems, and deep learning based signal reconstruction methods are starting to be used in important imaging technologies, for example in GEs newest computational tomography scanners and in the newest generation of the iPhone.

This course gives a graduate/master level introduction to deep learning based imaging. The course first introduces classical approaches to solving inverse problems and then aims to explain the recent advances of deep neural network based approaches for solving inverse problems in the imaging sciences.

Topics include classical sparse models, optimization for fitting classical methods and for training deep networks, unrolled algorithms, convolutional neural networks for image recovery and generation, generative models for image recovery, un-trained neural networks for signal recovery. The course ends with a brief outlook on how to apply those methods beyond image recovery for the recovery of a variety of other signals.

Upon successful completion of the module, students will be able to i) apply learning based methods to inverse problems such as signal and image recovery from few and noisy images, ii) evaluate their theoretical foundations and limits, and iii) critically evaluate papers and methods in that area, and iv) design variations of existing methods.

Analysis, an introduction to probability and statistics, and linear algebra. An introduction to machine learning is very helpful, but not necessary.

The course will take place in two sessions on the same day: In the first session the foundations of a topic are explained during lectures, with a focus on methods, ideas, and the theory behind the methods.
In the second session, we will discuss the method in more detail, go through a problem, or discuss a recent paper on that topic. If we discuss a paper, all students will need to read that paper before class so that we can have a meaningful discussion.

The lecture will be given via zoom. Lecture notes and exercises will be distributed.

Lecture notes will be distributed.

Students take a written exam of two hours duration. The exam consists of questions on the theory and algorithms of signal recovery with deep neural networks. The exam tests whether students can analyse, evaluate, and design solvers for inverse problems with deep networks. The exam is open-book, thus lecture notes and any other notes are permitted, but no computational devices are needed or are allowed.

Besides the written exam, 20% of the grade will be either through a presentation of a paper or through homework submission. In either case we will test whether students can analyse, evaluate, and design solvers for inverse problems with deep networks.
Specifically, if sufficiently few students are enrolled, each student has to present or defend a paper in the discussion session, and the evaluation of the paper, as well as the quality of presenting the arguments will count 20% towards the final grade.
If more students are enrolled than enabling each student to present a paper, then students will not present papers. In this case the submission of the homeworks which will contain evaluation and design of solvers for inverse problems will count 20% towards the final grade. The homework will contain analysis, evaluation, and design problems on solving inverse problems with deep networks.
The mode will be determined before the first lecture, and be communicated during the first lecture.