"A16-005 - Three-Dimensional Imagery Presentation from Multiple Aircraft Sensors"

The U.S. Army seeks to support Mission Commander (MC) situational awareness (SA) and decision-making by developing 3D presentation software, fusing full-motion video (FMV) and related metadata into a unified, high utility, display. Extending that to the realm of operational SA, to properly execute the OODA (i.e., Observe, Orient, Decide, and Act) loop, a comprehensive human-machine interface (HMI) is needed to aggregate data from existing archived data, online sensors, etc. This unified SA view will present accurate mission information in a relevant context, and provide decision-making technologies that offload the cognitive burdens presently faced by aircrews during operations, especially in complex and hostile environments. This is the primary goal, for instance, of the Army Degraded Visual Environment Mitigation Office. Our initial Phase II focused on combining information from multiple FMV sensors to generate a single unified 3D visual representation of a test “battlefield” using innovative probabilistic registration and 3D geometry building techniques. Initially named Micasa™, our software demonstrator also showcased innovation in real-time optimization, change detection and tracking, data model reduction, and user interface development. Building on our initial success, the Sequential Phase II herein proposes to extend the Micasa work with further emphasis on the following project components: (1) develop an operational automation of 3D geometry from FMV; (2) use existing point cloud information (e.g., from such sources as LIDAR or stereo-derived) with new 3D source information; (3) improve polygonization for exploitation, delivery, and archival purposes; and (4) fuse additional data, and provide data layering, for immersive HMI visualization. These points provide the basis for our stated objectives and the associated tasking plan herein, and they are balanced relative to the scope and level of effort expected from a 24-month Phase II.