Robot learning is currently being reshaped by foundation models: large-scale pretraining, generalist policies, and the integration of vision-language models with robot control. This seminar provides a structured introduction to modern Vision-Language-Action (VLA) systems and generalist robot policies, and focuses on the practical design choices that determine whether such models work reliably on real robots. We will start from the basics of modern imitation learning (behavioral cloning) and cover key policy families for robot manipulation, including action chunking transformers and diffusion/flow-based policies. Building on this, we will discuss what matters in learning from demonstrations, with an emphasis on data quality, action representations and tokenizers, and the trade-offs between autoregressive and generative (diffusion/flow) action generation. We will also introduce recent ideas for robust adaptation, such as modular design and knowledge insulation, and give a light overview of how experience-based fine-tuning can further improve performance.
| Rhythmus | Tag | Uhrzeit | Format / Ort | Zeitraum |
|---|
| Modul | Veranstaltung | Leistungen | |
|---|---|---|---|
| 39-M-Inf-AI-app-foc_a Applied Artificial Intelligence (focus) Applied Artificial Intelligence (focus) | Applied Artificial Intelligence (focus): Seminar | Studieninformation | |
| Applied Artificial Intelligence (focus): anwendungsorientiertes Seminar 1 | Studienleistung
|
Studieninformation | |
| Applied Artificial Intelligence (focus): anwendungsorientiertes Seminar 2 | Studieninformation | ||
| - | benotete Prüfungsleistung | Studieninformation | |
| 39-M-Inf-ASE-app-foc_a Applied Autonomous Systems Engineering (focus) Applied Autonomous Systems Engineering (focus) | Applied Autonomous Systems Engineering (focus): Seminar | Studieninformation | |
| Applied Autonomous Systems Engineering (focus): anwendungsorientiertes Seminar 1 | Studieninformation | ||
| Applied Autonomous Systems Engineering (focus): anwendungsorientiertes Seminar 2 | Studieninformation | ||
| - | benotete Prüfungsleistung | Studieninformation | |
| 39-M-Inf-INT-app-foc_a Applied Interaction Technology (focus) Applied Interaction Technology (focus) | Applied Interaction Technology (focus) - Seminar | Studieninformation | |
| Applied Interaction Technology (focus): anwendungsorientiertes Seminar 1 | Studienleistung
|
Studieninformation | |
| Applied Interaction Technology (focus): anwendungsorientiertes Seminar 2 | Studieninformation | ||
| - | benotete Prüfungsleistung | Studieninformation |
Die verbindlichen Modulbeschreibungen enthalten weitere Informationen, auch zu den "Leistungen" und ihren Anforderungen. Sind mehrere "Leistungsformen" möglich, entscheiden die jeweiligen Lehrenden darüber.
The seminar will start with two introductory lectures. Each week, one student will present a research paper from the reading list and lead the discussion. At the end of the semester, each student has the option to submit an essay surveying related literature to obtain extra credit.