Saliency Annotation of Image and Video Data

ABSTRACT: Toyon Research Corporation and the Pennsylvania State University propose to develop a software tool which allows an operator to interactively identify suspicious activities. The tool will be called ALARM (Active Learning for Anomaly Recognition and Mensuration). It will ingest a database of track data and automatically cluster the data and develop statistical models of the tracks. The statistical models will be used to rank the tracks in terms of the most anomalous to the least anomalous. The operator will observe the most anomalous tracks and use contextual information to identify the most suspicious tracks. The ALARM inference algorithms will use the labels provided by the operator to classify the labeled tracks as suspicious and to identify unlabeled tracks that may or may not have been classified as anomalous, and present these tracks to the operator for further labeling and classification. The track database that we will use during Phase II will be derived using stored data collected by the ARGUS-IS sensor. We will improve and run automated detection and tracking algorithms on the ARGUS-IS data to create the tracks. BENEFIT: The successful completion of the proposed research will allow analysts to effectively sift through large volumes of video and track data in short periods of time. This will be made possible by making use of low-level automation for video processing and tracking combined with high-level reasoning by humans to recognize what is suspicious and what is not. A classifier that treats as inputs features derived from the track measurements will be actively learned both to automatically predict whether given tracks are suspicious or not and to rank all tracks for prioritized forwarding to a human analyst for labeling.