TRIPWIRE: Threat Recognition, Identification and Prioritization with Infrared and Electro-optical Sensors

In combat scenarios soldiers are commonly tasked with detecting, recognizing and identifying threats to make well-informed tactical decisions. Even in well-controlled environments this can be a challenging task due to long distance to target, occlusion, and poor lighting conditions. To perform accurate threat assessment an automated system must be able to identify enemy capability, infer intent and predict the overall threat posed by persons and objects in proximity of the camera. These characteristics are typically not directly measurable and are often expressed in subtle observable cues. Our approach estimates threat level for detected targets using a multi-cue machine learning based approach. The proposed method extracts features from electro-optical and infrared imagery to identify personnel, weaponry and other objects in view of the camera. Several machine learning based detection, recognition, identification and context estimation modules contribute features which are fused and aggregated over time to describe long-term trends. These aggregated threat profiles are then classified to identify anomalies. Ultimately, the application of artificial intelligence (AI) and machine learning (ML) algorithms to monitoring threats in the battlefield and near civil infrastructure will improve the speed and probability of threat mitigation and increase safety for warfighters and civilians alike.