This lecture offers a journey through recent approaches towards a deeper theo-retical understanding of deep neural networks, focussing on questions such as properties of deep nets near initialization, mean field theory, complexity results for deep networks, properties of loss landscapes and gradient learning, infinite width limit of learning, neural tangent kernel, and renormalization group ap-proaches for the non-gaussian regime. Many results will be seen to stem from an analysis of certain limiting cases of neural network architectures (linear, vanishing depth/width ratio) that provide insights about properties of real net-works that are "close" to such cases.
The lecture will proceed along a "theory backbone" provided in the recent book "The principles of deep learning theory" by Roberts, Yaida and Hanin (2021), exploiting methods from theoretical physics for analyzing properties of deep networks which reveal striking analogies with phenomena in physical many-particle systems. This backbone will be complemented with selected papers that deepen the topics listed above further.
Prerequisites are a solid background in linear algebra, multivariate calculus and rudimentary probability theory.
Rhythmus | Tag | Uhrzeit | Format / Ort | Zeitraum | |
---|---|---|---|---|---|
wöchentlich | Do | 14-16 | X-B2-101 | 09.10.2023-02.02.2024
nicht am: 28.12.23 / 04.01.24 |
Lecture starts on Thursday, Oct-19 2023 |
Verstecke vergangene Termine <<
Modul | Veranstaltung | Leistungen | |
---|---|---|---|
39-M-Inf-AI-adv-foc Advanced Artificial Intelligence (focus) | Advanced Artificial Intelligence (focus): Vorlesung | benotete Prüfungsleistung
|
Studieninformation |
39-M-Inf-VML Vertiefung Maschinelles Lernen | Vertiefung Maschinelles Lernen | Studieninformation |
Die verbindlichen Modulbeschreibungen enthalten weitere Informationen, auch zu den "Leistungen" und ihren Anforderungen. Sind mehrere "Leistungsformen" möglich, entscheiden die jeweiligen Lehrenden darüber.
Zu dieser Veranstaltung existiert ein Lernraum im E-Learning System. Lehrende können dort Materialien zu dieser Lehrveranstaltung bereitstellen: