Urbanscape: Single Shot Multi-Task 3D Reconstruction

Hazard assessment tools that model the transport and dispersion of Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE) materials through urban areas are only as good as the 3D models that inform the physics model. Maintaining accurate, up-to-date 3D models of urban areas is challenging. Even in the commercial world, urban construction and demolition may result in the models created are often out of date for the simple reason that those models are created with LIDAR point clouds or point clouds derived from multi-view aerial or satellite imagery. These data sets are costly to curate and maintain. Recent advances in the area of Deep Learning based 3D reconstruction may be adapted and extended to achieve Urbanscape 3D models from single view satellite imagery. DZYNE’s Urbanscape system will employ deep neural networks to achieve pixel level depth estimation capabilities with semantic and geometric association based on a multi-task learning pipeline that exceed the results of any single task. 3D data annotation scarcity is overcome by training CNNs with a combination of 3D models, DSM, DEM, and multi-view satellite images. A depth map enhanced with semantic labels and geometric properties is created in the format that can be readily incorporated with CBRN tools.