Large scale semantic 3D modeling of the urban landscape

Modeling and understanding large urban areas is becoming an important topic in a world were everything is being digitized. A semantic and accurate 3D representation of a city can be used in many applications such as event and security planning and management, assisted navigation, autonomous operations, city surveying or urban planning to name just a few. Different sources of information can be used for this purpose. Cameras at the aerial or ground level, laser scanners, satellite imagery or geographical information data are common and widely used. In particular images based approaches have received a lot of attention given the flexibility and low cost of the cameras used to record the scene.
Given the complexity of the complete process, difficulties arise mainly in three stages: the decomposition of the image based reconstruction process, the accurate estimation of the camera positions and the semantic interpretation of the resulting model.
This thesis deals with the problem of large scale city-size reconstruction and modeling using a monocular camera. The goal of the research was twofold. Firstly, to obtain an accurate, fast and inexpensive method to perform 3D reconstruction. Secondly, to obtain a semantic model of the reconstructed environment.
We decompose the reconstruction procedure and design a processing pipeline for 3D reconstruction of urban areas by exploring the range of algorithms and methodology choices. By careful reasoning and comparison of state-of- the-art methods we are able to optimize the results of the algorithms involved. We propose an algorithm for estimating optimally, and in closed form, the scaled translation of a camera with as little as one correspondence between the 3D space and the 2D image space. Finally, we approach the problem of semantic modeling of large urban areas by merging information from different sources to reach a detailed building level description.