392153 Visual Computing (V) (SoSe 2026)

Our world is inherently non-rigid at different spatial and temporal scales. Reconstructing and modelling it in 4D from visual observations is a vibrant research field that remains challenging and that has numerous practical applications, for instance, in AR/VR/XR, computer game development, human-computer interaction and sport analytics. The frequent challenges of this field include the ill-posedness of the underlying optimisation problems and settings (e.g., monocular, which is of special interest in Computer Vision) and observed scene conditions (e.g., partial observations, low light or high-speed motions), among many others. The goal of the lecture "3D and 4D Computer Vision" is to introduce foundational concepts of 3D computer vision for deformable and composite scenes (4D = 3D + time) as well as results of the latest research in the field in a systematic and structured manner through generalisation of studied concepts from 3D to 4D cases.

The lecture will cover the fundamentals of 3D computer vision applicable across a wide range of 3D and 4D settings (multiple view geometry, triangulation, stereo vision, bundle adjustment, linear transformations, parametrisations of rotations), different types of visual sensors (RGB, event and depth cameras), 3D and 4D scene representations, deformation models and regularisers, non-rigid structure from motion (NRSfM), shape-from-template, correspondence problems, novel-view synthesis of non-rigid scenes, generative and diffusion models in 4D vision, 3D human pose estimation, egocentric 4D vision as well as video generation of composite scenes. Apart from milestone methods in the field, the lecture will discuss several recent works on 4D vision including state-of-the-art approaches.

Covered Topics:

• Fundamentals of 3D and 4D Computer Vision
• 3D and 4D Scene Representations
• Correspondence Problems in 3D
• Multi-View Geometry, Structure from Motion and Bundle Adjustment
• Monocular and Depth-Based 4D Reconstruction
• 3D Volumetric Rendering of Rigid and Non-rigid Scenes
• 3D Human Pose Estimation
• Egocentric 4D Vision from Mobile Head-Mounted Devices
• 3D and 4D Generative and Diffusion Models
• Controllable Video Generation for Non-rigid Scenes
• Event-based 3D and 4D Vision
• Recent 4D Vision Research and Research Trends

Requirements for participation, required level

The course builds upon the programming skills, mathematical models, and concepts learned in the introductory Programming and Math courses.
Required are basic programming skills (in python or Julia) and basic mathematics (in particular linear algebra, some integral calculus, little probability theory).

Prior knowledge of the PyTorch framework typically used for machine learning is a plus but not required.

Bibliography

The course material is largely self-contained and refers to external sources where not. The https://scratchapixel.com/ website is an excellent resource for self study of the covered materials and beyond. A script covering the core methodologies may be provided in future iterations of this course.

External comments page

https://docs.google.com/presentation/d/e/2PACX-1vRGyE0j_cU324qp-vOEE4hEBPn3HdwliZQqeg_vT0w7Yt9EUXmgFB6NamL_ynqeBMYRgD-cdvfvXobA/pub?start=false&loop=false&delayms=3000