ARCHIVE: Adaptive Responses to Context and History in Video Exploitation

Archive enables improved exploitation of persistent wide-area sensors. These sensors offer new capabilities for maintaining battlefield situational awareness, but realization of their benefits is blocked by bandwidth limitations that prevent sending all the imagery to the ground, and by human limitations that slow interpretation of large data volumes. In Archive, foveated (variable resolution) imagery is used to reduce bandwidth while maintaining operational effectiveness. Acquisition and tracking algorithms localize and highlight targets, and drive the placement of high-resolution foveae. Consumers receive regions of interest at full fidelity while a lower-fidelity periphery maintains their situational awareness. The key innovation in Archive is a set of novel foveated exploitation techniques that improve performance by exploiting the long-term history unique to persistent wide-area sensors. Acquisition and tracking are improved by exploiting historical scene appearance, target position, target behavior, algorithm behavior, geospatial information, 3D structure, time of day, and other data. These provide richer statistical models, non-uniform priors, fewer false alarms, better behavior prediction, better track segment fusion, and improved situational awareness. Archive Phase 1 conducts experiments to demonstrate the feasibility of the approach. Archive leverages 21st Century’s unique expertise and capabilities to provide superior exploitation with lower communication bandwidth and lower operator workload. BENEFIT: The Archive Phase 1 and 2 efforts will generate an engineering prototype of a foveated exploitation system that will collect, analyze, and exploit historical and contextual information stored in a pixelpedia or voxelpedia. Archive will overcome two obstacles currently blocking the exploitation of wide-area persistent surveillance imagery on the battlefield: limits on communication bandwidth and operator attention. Archive supports improved operational performance by providing superior target acquisition, tracking, and situational awareness to the warfighter through improved exploitation of wide-area persistent sensors. Use of historical and contextual information yields lower false alarm rates and improved target behavior prediction, both of which reduce operator workload. Archive provides an exploitation approach that learns and adapts, improving its performance over time. As a result, each intelligence consumer can support a larger geographic region, support that region more effectively, and provide more beneficial intelligence products to the battlefield commander. Foveation enables transmission of large format wide-area imagery to the ground, providing partial relief to communication systems under increasing demands. Exploiting long-term history and context information in target acquisition and tracking will allow the creation of spatially- and time-varying prior target probabilities for both position and behavior, thereby reducing false alarm rates and improving target state estimates and track segment fusion. These are novel extensions to the state of the art in foveated image exploitation systems, as well as the first examples of an extensible new paradigm for improving image exploitation through the use of history and context stored in the form of a 2D pixelpedia or 3D voxelpedia. Phase 1 provides a quantitative evaluation of improvements to the performance of acquisition and tracking algorithms achieved by the approach. Phase 1 also provides the first three (of many) specific examples of how history and context can be encoded in a pixelpedia and exploited to improve performance. First, we use a rich understanding of background appearance history to reduce acquisition false alarm rate. Second, we use historical knowledge to build non-uniform target prior probabilities, again improving acquisition performance. Finally, we use historical knowledge of common target behavior to improve target behavior prediction and tracking. Underlying these three strategies is the construction of our first embodiment of the pixelpedia concept. Together, these three specific improvements form a solid proof of the Archive concept and demonstration of its feasibility. Detailed Phase 1 results will also provide insight into which facets of history and context exploitation are most promising, allowing our Phase 2 effort to be shaped for maximum technical and operational benefit to the warfighter.