392272 Projekt: 6D Object Pose Estimation with 3D Geometry Foundation Models (Pj) (WiSe 2026/2027)

IMPORTANT: please contact the project supervisor for participation: dmitrii.trofimov@uni-bielefeld.de

Recent 3D geometry foundation models such as VGGT (CVPR 2025 Best
Paper) take a handful of photographs from unknown camera positions and predict
the 3D structure of the scene together with all camera poses in a single forward
pass. We use such a model to estimate the pose of objects it has never seen
during training, from a few reference photographs alone.

This works when the object is large and clearly visible, and degrades when it is
small, occluded, or photographed under conditions unlike the training data.
Possible directions include:

• predicting which additional camera viewpoint would help most, before the

picture is taken

• measuring which differences between rendered and real images actually cost

accuracy, and building training data that closes the gap

• asking the model what it would see from a viewpoint at which no picture was

taken

• quantifying how difficult the standard benchmark datasets really are

Requirements for participation, required level

Solid Python skills and comfort working with an existing PyTorch codebase are
expected. An interest in 3D data and in reading research papers is helpful;
prior experience with computer vision or robotics is not required — the
necessary background is introduced in the first weeks.

Bibliography

• Wang et al., VGGT: Visual Geometry Grounded Transformer, CVPR 2025 —

https://arxiv.org/abs/2503.11651

• Wang et al., DUSt3R: Geometric 3D Vision Made Easy, CVPR 2024 —

https://arxiv.org/abs/2312.14132

• Xu et al., ''Pose Anything Anywhere: Model-free Object Poses from Arbitrary

References'', ECCV 2026 — https://arxiv.org/abs/2606.23634

• Wen et al., ''FoundationPose: Unified 6D Pose Estimation and Tracking of Novel

Objects'', CVPR 2024 — https://arxiv.org/abs/2312.08344

• ''BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose

Estimation'' — https://arxiv.org/abs/2504.02812