392150 Reinforcement Learning in the Physical World (S) (WiSe 2026/2027)

Reinforcement learning ultimately promises agents that continuously learn from their own experience and adapt to novel goals and environments. Yet most learning systems that act in the physical world, from robot arms to legged robots and vehicles, are trained once on a fixed dataset and then deployed without ever learning again.

Letting such systems keep improving from real interaction raises challenges that are largely hidden in simulation. Real world data is slow and expensive to collect, which makes sample efficiency and the reuse of prior data central. Behavior has to be represented by expressive policies over continuous, high dimensional action spaces, and these policies must be improved without losing what they already know. Exploration has to stay safe, learning has to proceed autonomously without manual resets or hand engineered rewards, and decisions have to be made in real time despite inference latency. And because the world itself keeps changing, learning should never really stop.

This seminar works through recent research on these challenges: generative and action chunked policy classes, offline and offline to online reinforcement learning, policy improvement for diffusion and flow policies, human in the loop learning and reinforcement learning on top of large pretrained models, reset free learning, inference latency, safe and model based reinforcement learning, and finally the case for continual reinforcement learning during deployment. Most examples come from robotics, but the questions apply to any learning system that has to act in the real world.

After three introductory sessions, each participant presents one recent research paper (25 to 30 minutes) and leads the following discussion. Participants taking the graded variant additionally write an essay (10 to 15 pages) on a research question related to their paper. Topics are assigned in the first weeks via Moodle.

Requirements for participation, required level

Recommended: solid basics in machine learning and deep learning (neural networks, loss functions, gradient based optimization), basic probability and linear algebra. Prior experience with reinforcement learning or robotics is helpful but not required; the introductory sessions cover the necessary background. You should be comfortable reading current research papers in English.