392127 Opening the Black Box from the Top, Bottom, or Something In-between? (S) (WiSe 2026/2027)

As AI systems grow more capable and are deployed in increasingly high-stakes contexts, understanding why they behave the way they do is no longer optional; it is a prerequisite for building systems we can trust. But how do you open a black box? Do you start from the top, reasoning about high-level concepts and patterns? Or do you dig from the bottom, tracing the precise computations happening inside individual neurons and circuits? This seminar will explore both angles.

Specifically, we will investigate two complementary branches of AI interpretability through the lens of safety. Concept-based Interpretability takes a top-down approach, probing a model's learned representations for human-interpretable concepts — and using those concepts to steer model behavior toward desirable properties like honesty and harmlessness. Mechanistic Interpretability takes a bottom-up approach, reverse-engineering the internal circuitry of neural networks to identify the precise computational structures responsible for specific behaviors. Together, these approaches form much of the latest research in understanding and shaping AI behavior.

In this seminar, students will drive the learning process by presenting recent papers across both approaches, demonstrating key methods, and leading group discussions. Towards the end of the semester, students will apply what they have learned in a hands-on project. Each student (or group) will select a model exhibiting a safety-relevant behavior they wish to change, implement either a concept-based or mechanistic intervention, and defend their methodological choice — explaining what their chosen approach revealed and what the alternative would or would not have uncovered.

Requirements for participation, required level

Knowledge of Machine Learning / AI

Bibliography

literature will be provided before the seminar starts