You are here

Enhancing Motion with Foundation


TECHNOLOGY AREA(S): Electronics, Information Systems, Sensors


Develop and demonstrate a capability to generate narrative descriptions and structured summaries of events, activities and anomalies associated with locations from mover intelligence (MOVINT), Geographic Information Systems (GIS) and contextual foundation data.


Existing sources of MOVINT are generating massive amounts of persistent data of fixed locations, and more platforms are being planned. Detecting and tracking movers in full motion video (FMV), wide area motion imagery (WAMI), moving target indication (MTI) data, and other MOVINT sources have developed mature capabilities deployed for various platforms. However, converting tracks into meaningful intelligence has received relatively little attention beyond manual analysis and summary visualization techniques such as heatmaps of traffic density. Automated track analytics, such as complex threat and anomaly detection, have been hampered by short track durations, particularly in urban areas; intermittent coverage, leading to significant temporal gaps at arbitrary times; and the difficulty of incorporating higher-level, semantic understanding of the scene and cultural behaviors.

This topic will develop methods to automatically detect significant activities, anomalies and relationships from MOVINT and use them to produce human-level, semantic summaries of the most salient information associated with a location, facility or other fixed entity. GIS information from foundation feature databases should be incorporated to provide prior knowledge of the scene in the form of known buildings, facilities and structures. The interactions and relationships of movers to those features should be explicitly incorporated into algorithms to provide context sensitivity and semantic understanding that would be useful to an analyst responsible for monitoring the scene. For a designated area and temporal interval, the methods should produce an activity summary that includes structured information such as the most significant, unusual or salient events, and a narrative, textual description of that information in natural language text. Ideally, an analyst would be able to delve into any part of the summary to examine the intermediate layers of information, such as individual events, locations, and underlying raw data used to discover them.

The methods should scale to city-size areas with hours of coverage per day, enabling an analyst to rapidly obtain an automated summary of any specified region of the scene such as a single building, a parking area, a compound or a city block. Summaries should highlight activities that are unusual or significant within the local cultural context, such as high amounts of activity at a religious facility when it is not the normal time for ceremonies there, or no activity when there should be a ceremony there. The system should not rely on data-driven methods to learn patterns of life, but instead should infer expected behaviors and other information from prior cultural knowledge encoded in a suitable representation.


Using WAMI data, show the feasibility to generate summaries of salient events at a designated location, emphasizing the improvement in salient activity detection and summarization from leveraging GIS and cultural information. Phase 1 will provide an initial proof of concept using constrained spatial and temporal information to create structured representation summaries.


Develop a mature algorithmic capability implemented within a prototype to generate salient summaries, both structured and narrative, of arbitrary regions across multiple scales, multiple MOVINT data types and multiple cultures. GIS and cultural information should be encoded in structured representations and leveraged for inference about important activities vice benign or insignificant ones. The prototype should provide a user interface for analyst evaluation of the system on operationally relevant data.


Fully develop and transition the technology and methodology based on the research and development results developed during Phase II for DOD applications in the areas of MOVINT analytics, and other anomaly surveillance and reconnaissance applications. For example, civil authorities might use MOVINT for disaster relief, or transportation monitoring

KEYWORDS: full motion video (FMV); wide area motion imagery (WAMI); moving target indication (MTI) data

US Flag An Official Website of the United States Government