KATHLÉN KOHN

The primary goal of this project is to develop the theoretical foundations for 3D scene reconstruction from images taken by unknown rollingshutter cameras, which is the overwhelming camera technology of today. Implementing sufficiently fast reconstruction algorithms for rolling-shutter cameras – without restricting assumptions – is a major open challenge in computer vision.

Algebraic geometry provides the natural tools for rigorous theoretical foundations for that challenge, yet the algebraic community has not investigated rolling-shutter cameras.

From the algebro-geometric perspective, rolling-shutter cameras are parametrizations of algebraic surfaces in the Grassmannian of 3D lines, and 3D reconstruction amounts to computing fibers under rational maps. AVIS will use this inherent geometry to find exhaustive lists of efficiently solvable algebraic reconstruction problems (so-called minimal problems) and develop new intersection-theoretic tools to measure their intrinsic complexity.

This project (AVIS) will describe the critical loci of reconstruction where problem instances are ill-conditioned and prone to numerical instability. Moreover, AVIS will compute the polynomial constraints encoding relative camera poses, and calculate geometric invariants that capture the complexity of triangulation (i.e., the problem of recovering 3D coordinates from known cameras and 2D data) under noise.

If successful, the algebro-geometric foundations developed in this project will lead to the implementation of fast 3D reconstruction algorithms with rolling-shutter cameras. AVIS’ unified and abstract view will yield tools applicable to 3D reconstruction in general, beyond rolling-shutter cameras, and lead to long-lasting foundational results in modern computer vision. The mutual exchange between algebraic geometry and computer vision will open new horizons for scholarship in both fields.