You are here

Transforming 3D Reconnaissance Data into Geospatial Intelligence

Description:

TECHNOLOGY AREA(S): Info Systems 

OBJECTIVE: To design, analyze, and implement new algorithms and a software system for streaming and processing 3D reconnaissance data for enabling large-scale Unmanned Aerial System (UAS) operations in the low altitude urban-suburban airspace. 

DESCRIPTION: Current Army, DoD, and civilian capabilities for site exploration missions in urban-suburban areas, especially reconnaissance and rescue operations, are inaccurate,heavy, expensive, dangerous, and time consuming. Unmanned Aerial Systems (UAS)could potentially provide real-time military reconnaissance, fire and rescue, law enforcement, and other first-responders with important new ways to enhance mission effectiveness and reduce operational costs. While the small Unmanned Aerial Vehicles needed for such missions are now available at reasonable cost, the navigational and control systems and associated software required to conduct such coordinated, precision autonomous operations in low altitude urban and suburban airspaces are not yet available. This is because current state-of-the-art systems rely heavily on the U.S. NAVSTAR global positioning system (GPS) and global navigation satellite system (GNSS). However, in low altitude urban and suburban airspaces the high density of obstacles and the presence of people necessitate a degree of navigational precision and reliability that cannot be met by GPS, which can have limited precision near buildings, or existing “sense and avoid” technologies. Software tools and algorithmic techniques not dependent on GPS are necessary for UAS navigation in the urban-suburban airspace. One such technique is three dimensional (3D) map-matching. 3D map-matching is a navigational basis that is orthogonal to radio navigation and consequently does not suffer from the same limitations and vulnerabilities of GPS. Early 2.5D map matching systems such as TERCOM (Terrain Contour Matching), were effectively employed in cruise missile navigation prior to GPS. The ability to pre-acquire detailed 3D geospatial data has increased exponentially since the time of TERCOM. Moreover, commodity sensors are now available which generate real-time point-clouds that could potentially be matched to the pre-acquired 3D geospatial data to provide rapid, precise localization in many GPS denied environments. However, two problems have slowed the evolution of efficient 3D map-match solutions. First, because the 3D geospatial data sets are so large, it can be difficult to transmit and maintain them over bandwidth and latency constrained networks using conventional data delivery approaches. Second, processing of these massive 3D datasets by 3D map-matching algorithms can be very inefficient because the matching algorithm is typically forced to process a large amount of occluded data that is irrelevant to the immediate 3D map-match localization solution. This is especially true in densely occluded natural terrains or within the urban canyon. The ultimate goal is the design of algorithmic techniques resulting in a software system that can overcome the delivery and processing problems of 3D map-matching and efficiently stream 3D reconnaissance data over constrained networks and use this data to perform precise localization for UAS to navigate in suburban and urban terrains. This software system should be able to encode these massive 3D data sets or some subset sufficiently necessary for navigation purposes, including geometric visibility, of previously obtained 3D maps of the urban terrain and efficiently transmit this data to the UAS navigational system in real time. Then the system should be able to match the current sensor-derived ground truth obtained by the UAS sensors to the streamed 3D representation, also in real time, to enable instant, on-demand access to timely and detailed 3D data for analysis, mission planning, mission rehearsal, and battle damage assessment. Besides enhancing military operations, such a system would have a wide variety of civilian uses such as fire and rescue, law enforcement, and other first-responder situations making it highly viable as a commercial product. Such software could easily be licensed for both military and civilian purposes or marketed as a single software package. 

PHASE I: This portion of the effort will consist of identifying robust and mathematically consistent computational approaches to stream 3D reconnaissance data and perform precise localization for UAS navigation. This can be accomplished by (1) investigating and recommending or developing efficient techniques to stream massive 3D data sets of previously obtained 3D maps of the urban terrain to the UAS navigational system in real time and (2) investigating and recommending or developing appropriate techniques to match sensor-derived ground truth to the streamed 3D representation, also in real time. Then conduct a proof-of-concept simulation of each of the above. 

PHASE II: Using the results from Phase I, the effort will be to build a robust, scalable software system for streaming 3D reconnaissance data and perform precise localization for UAS navigation. This can be accomplished by (1) implementing the technique from Phase I to stream massive 3D data sets of previously obtained 3D maps of the urban terrain to the UAS navigational system in real time, (2) implementing the technique from Phase I to match sensor-derived ground truth to the streamed 3D representation, also in real time, and (3) incorporating the above into a single software system. In addition, a comprehensive set of software documentation will be prepared and made available for users and a long-term program for maintenance and subsequent improvement of the software will be created. 

PHASE III: The outcome of this effort would be the development of a software system for transforming and streaming 3D reconnaissance data and performing precise localization for UAS navigation that contains significantly more information than video, but which requires less bandwidth. By combining sensor-based, data driven navigation and efficient continuous remapping, this effort could realize a scalable, sustainable, and deliverable representation of any environment and enable important new capabilities in autonomous navigation and intelligent tactical maneuvering. Consequently, this effort could increase the speed and reduce the cost of processing, exploiting, and disseminating 3D geospatial data for both military and civilian operations in urban and suburban settings such as reconnaissance, fire and rescue, law enforcement, and other first-responder activities. The firm will follow-up on appropriate marketing and licensing opportunities from collaborations and contacts developed during earlier phases. The company will set up a support service for both existing and new users capable of addressing installation issues and correcting bugs. This will include creating a web site with the latest news, FAQs, user' forum, etc. 

REFERENCES: 

1: P. Agarwal and R. Sharathkumar, "Streaming algorithms for extent problems in high dimensions," Proc. 21st Annual ACM-SIAM Symposium on Discrete Algorithms, 2010.

2:  C. Poullis and S. You, "3D Reconstruction of Urban Areas," Proc. of IEEE 3D Imaging, Modeling, Processing, Visualization, and Transmission (3DPVT), May 2011

3:  J. Huang and S. You, "Point Cloud Matching based on 3D Self-Similarity," Proc. of IEEE CVPR Workshop on Point Cloud Processing, June 16, 2012

4:  Stump, Ethan, et al. "Visibility-Based Deployment of Robot Formations for Communication Maintenance" ICRA, IEEE Intel. Conference, 2011

5:  Ji Zhang and Sanjiv Singh, "LOAM: Lidar Odometry and Mapping in Real-time," Robotics: Science and Systems Conference, July, 2014.

KEYWORDS: 3D Map-matching, 3D Reconnaissance Data, Streaming, Unmanned Aerial Systems Navigation, Low Altitude Urban Airspace, GPS Denied Environment 

CONTACT(S): 

Joseph Coyle 

(919) 549-4256 

joseph.michael.coyle@us.army.mil 

US Flag An Official Website of the United States Government