You are here

Long-distance 3-D Reconstruction from EO/IR Imagery


OBJECTIVE: To combine machine learning-based single image three-dimensional (3-D) reconstruction techniques with current multiple image techniques, enabling long-distance, low-angular diversity 3-D reconstruction. DESCRIPTION: Recently, significant gains have been made in the capability to generate 3-D models of an area from aerial electro-optical or infrared (EO/IR) imagery. The ability to create a 3-D model of the observed area is beneficial for many downstream image exploitation applications. For example, change detection, target tracking, and geo-registration all benefit from the information contained in a accurate 3-D model of the area being observed. In addition, a 3-D reconstruction of an area of interest can help the warfighter to quickly obtain an understanding about that area. While current techniques for 3-D reconstruction from EO/IR imagery are starting to yield high-quality 3-D models, the algorithms used to implement these techniques require significant angular diversity between the imagery. In other words, accuracy of the model is dependent on collecting imagery from multiple viewpoints of a single object, with large differences in viewing angle present among the set of viewpoints. This requires a flight pattern that enables the aircraft collecting the data to observe the area for an extended period of time, from multiple viewpoints. This type of flight pattern is not feasible in many scenarios, requiring a modified approach to 3-D reconstruction. In addition to angular diversity-based 3-D reconstruction approaches, algorithms have been introduced in the research literature that utilize machine learning techniques to perform 3-D reconstruction from single images (e.g., see references below). These techniques, however, generally ignore the possibility of multiple images of an object. The Air Force seeks techniques that merge both multiple image (angular diversity) and single image reconstruction techniques to enable long-distance 3-D reconstruction of an area. In long-distance reconnaissance scenarios, there will not be enough angular diversity between collected images to enable high-accuracy multiple image reconstruction techniques in isolation. However, improved accuracy over single-image reconstruction techniques should be possible with multiple images collected of the same area with some small amount of angular diversity. Techniques that optimally fuse both single and multiple image 3-D reconstruction methodologies are the focus of this work. Data from an aerial platform collecting imagery of 3-D objects at long range will be the target input for this work. Commercialization potential: Approaches developed under this project could be used in the processing chain of any military surveillance asset. In addition, non-defense related applications include the automated mapping of urban areas (e.g., for Google or Bing maps), civil surveying, and the automated creation of virtual environments. PHASE I: In Phase I, the contractor will demonstrate 3-D reconstruction algorithms on a government-provided data set with views differing by less than 90 degrees. Improved performance will be demonstrated by combining single and multiple image reconstruction techniques. An accuracy comparison between reconstruction approaches at various distances from the target will be delivered. PHASE II: The contractor will expand the operating conditions under which accurate 3-D reconstruction can occur, including more limited angular diversity, different types of terrain, and allowing limited human intervention to quickly reconstruct large areas. Fusion with external inputs (e.g., Global Positioning Satellites (GPS), inertial measurement units (IMUs), geo-referenced satellite imagery) may also be enabled. Demonstrations using both EO and IR imagery will be performed. PHASE III: This technology will support the warfighter in quickly obtaining an understanding of areas that are observed by long-distance aerial platforms. Phase III will be used to take the technology developed and create a product that is robust and reliable enough for use in real-world scenarios. REFERENCES: 1. D. Hoiem, A.A. Efros, and M. Hebert,"Geometric context from a single image", IEEE International Conference on Computer Vision, 2005. 2. A. Saxena, M. Sun, and A.Y. Ng,"Make3D: Learning 3D scene structure from a single still image,"IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 31, Issue 5, 2009, Pages 824-840. 3. D.C. Lee and M. Hebert and T. Kanade,"Geometric reasoning for single image structure recovery", IEEE Conference on Computer Vision and Pattern Recognition, 2009. 4. M. F. Tappen, W. T. Freeman, and E. H. Adelson."Recovering Intrinsic Images from a Single Image". In IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 27, Issue 9, 2005, Pages 1459 - 1472. 5."Multiple View Geometry, 2nd Edition", Richard Hartley and Andrew Zisserman, 2003, Cambridge University Press.
US Flag An Official Website of the United States Government