P5. 3D Surface Reconstruction and Interpretation
Homepage:
http://vision.in.tum.de/research/robotvision/mavs
Project coordinator/staff: D. Cremers, W. Förstner, J. Stückler, V. Usenko, G. Kuschk
Summary:
The goal of this project is to estimate dense 3D reconstructions of the environment from images. The reconstruction will be based on the camera poses and surface patches provided by P1 and P4. We consider the main challenge in this project to be the computation of high-quality reconstructions of arbitrary shapes in real-time. To meet this challenge, we will identify and improve algorithms that can be used to solve this problem. While the overall goal of this project is to achieve an optimal 3D reconstruction from all views, it might be necessary to approximate the geometry by first computing local depth maps from multiple views and to fuse them efficiently. Such fusion methods are very fast and can be computed in real-time.
The second challenge of this project is to stabilize the estimated geometry over time. This means that we aim at iteratively integrating new observations in the previous estimate of the geometry to optimally exploit previous information with the goal to continuously increase the accuracy. To ensure real-time operation of our system, we plan to adapt the resolution of the reconstruction based on the scene content. For example, more processing power can be invested in the regions where additional detail is required. This will be implemented using the level-of-detail model and scene interpretation from P7 as well as the semantic segmentation from P4 and P5. Finally, we plan to estimate additional information about the surface, such as color and texture information. This information can then be used iteratively to further improve the surface reconstruction.