Saliency Annotation of Image and Video Data

Award Information
Agency: Department of Defense
Branch: Air Force
Contract: FA8650-11-M-1163
Agency Tracking Number: F10B-T15-0232
Amount: $100,000.00
Phase: Phase I
Program: STTR
Awards Year: 2011
Solicitation Year: 2010
Solicitation Topic Code: AF10-BT15
Solicitation Number: 2010.B
Small Business Information
6800 Cortona Drive, Goleta, CA, -
DUNS: 054672662
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: N
Principal Investigator
 Fritz Obermeyer
 (805) 968-6787
Business Contact
 Marcella Lindbery
Title: Director of Finance
Phone: (805) 968-6787
Research Institution
 The Pennsylvania State University
 Christine Wilson
 101 Hammond Building
Philadelphia, PA, 16802-
 (814) 865-3272
 Nonprofit college or university
ABSTRACT: With the current flood of surveillance data available to ISR analysts, human attention has become the most valuable resource to ISR systems. Although automated tracking and labeling algorithms are now capable of automatically identifying and roughly classifying targets, the current rate of false alarms and irrelevant annotations makes existing technology unsuitable for wide-area persistent surveillance applications, where analysts are overwhelmed by irrelevant data. What is needed is a system that incorporates a user-trainable relevance/saliency classification algorithm with the best available tracking algorithms to achieve very low clutter rates even in urban environments. Toyon Research Corporation and Penn State Professors David Miller and George Kesidis propose to address this need through a prototype system for automated saliency annotation, incorporating recent results in active learning of semisupervised mixture models, and automated feature extraction from video data with reconstructed 3D models. The proposed system combines the extremely low clutter rates of Toyon"s 3D clutter suppression algorithm, with high-accuracy classification methods using fine-grained mixture models developed by Professors Miller and Kesidis. BENEFIT: The capability generated by the proposed system will be crucial to Air Force ISR systems that rely on real-time processing and mining of wide-area persistent surveillance (WAPS) data, by dramatically increasing the effective surveillance region size per operator. The technology also reduces potential tactical cost in the consequences associated with misidentifying critical targets (either missing terrorist activity, or incorrectly targeting innocent civilians). Active learning has the potential both to greatly reduce the amount of labeling analysts need to do to achieve accurate automated classification/saliency determinations and, by achieving accurate classification, to reduce the risk of adverse tactical consequences. Additionally, the technology has many non-military applications, including manufacturing, construction, security, and automobile traffic monitoring, where the rarity of salient events limits the effectiveness of existing wide-area video systems from being effective, due to shortage of human resources.

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government