USA flag logo/image

An Official Website of the United States Government

Saliency Annotation of Image and Video Data

Award Information

Department of Defense
Air Force
Award ID:
Program Year/Program:
2011 / STTR
Agency Tracking Number:
Solicitation Year:
Solicitation Topic Code:
Solicitation Number:
Small Business Information
Toyon Research Corp.
6800 Cortona Drive Goleta, CA -
View profile »
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Phase 1
Fiscal Year: 2011
Title: Saliency Annotation of Image and Video Data
Agency / Branch: DOD / USAF
Contract: FA8650-11-M-1163
Award Amount: $100,000.00


ABSTRACT: With the current flood of surveillance data available to ISR analysts, human attention has become the most valuable resource to ISR systems. Although automated tracking and labeling algorithms are now capable of automatically identifying and roughly classifying targets, the current rate of false alarms and irrelevant annotations makes existing technology unsuitable for wide-area persistent surveillance applications, where analysts are overwhelmed by irrelevant data. What is needed is a system that incorporates a user-trainable relevance/saliency classification algorithm with the best available tracking algorithms to achieve very low clutter rates even in urban environments. Toyon Research Corporation and Penn State Professors David Miller and George Kesidis propose to address this need through a prototype system for automated saliency annotation, incorporating recent results in active learning of semisupervised mixture models, and automated feature extraction from video data with reconstructed 3D models. The proposed system combines the extremely low clutter rates of Toyon"s 3D clutter suppression algorithm, with high-accuracy classification methods using fine-grained mixture models developed by Professors Miller and Kesidis. BENEFIT: The capability generated by the proposed system will be crucial to Air Force ISR systems that rely on real-time processing and mining of wide-area persistent surveillance (WAPS) data, by dramatically increasing the effective surveillance region size per operator. The technology also reduces potential tactical cost in the consequences associated with misidentifying critical targets (either missing terrorist activity, or incorrectly targeting innocent civilians). Active learning has the potential both to greatly reduce the amount of labeling analysts need to do to achieve accurate automated classification/saliency determinations and, by achieving accurate classification, to reduce the risk of adverse tactical consequences. Additionally, the technology has many non-military applications, including manufacturing, construction, security, and automobile traffic monitoring, where the rarity of salient events limits the effectiveness of existing wide-area video systems from being effective, due to shortage of human resources.

Principal Investigator:

Fritz H. Obermeyer
(805) 968-6787

Business Contact:

Marcella R. Lindbery
Director of Finance
(805) 968-6787
Small Business Information at Submission:

Toyon Research Corp.
6800 Cortona Drive Goleta, CA -

EIN/Tax ID: 953526665
Number of Employees:
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Research Institution Information:
The Pennsylvania State University
101 Hammond Building
Philadelphia, PA 16802-
Contact: Christine Wilson
Contact Phone: (814) 865-3272