Chemical-Imaging Registration and Multi-modal Analysis

To address the Chemical and Biological Defense (CBD) agency’s need for a new deep learning (DL)-based threat detection solution that fuses imaging sensors and chemical/biological sensors to detect and locate concealed chemical threats, Intellisense Systems, Inc. (Intellisense) proposes development of a new Chemical-Imaging Registration and Multimodal Analysis (CIGMA) solution. CIGMA will comprise advanced onboard sensor data analytics software running on a commercial off-the-shelf (COTS) embedded accelerated computing platform (such as NVIDIA Jetson Nano) that fuses data from chemical and imaging sensors in real time for accurate detection, localization, and identification of concealed chemical threats. CIGMA consists of multiple DL-based image analytics tools and algorithms to detect containers of potential concealed chemical threats. By spatially registering the containers and chemical sensor readings, CIGMA can pinpoint the source of a chemical threat. Thus, the CIGMA system not only improves threat detection confidence but also differentiates chemical threat types (such as chlorine, tear gas/pepper spray, and nitrogen oxides). It also increases location accuracy of the chemical threats (better than 5-meter accuracy), significantly improving countermeasure capabilities. As a result, this system directly addresses the CBD requirements by offering improved threat detection confidence, identification, and location accuracy. In Phase I, Intellisense will demonstrate the feasibility of CIGMA by developing and testing its deep fusion architecture which jointly exploits inputs from imaging and chemical sensors for the detection, identification, and localization of concealed chemical threats. The performance of CIGMA will demonstrate improved detection/identification rates over single-mode DL automated detection algorithms on the discrimination of at least two or more chemical threats (such as chlorine, tear gas/pepper spray, and nitrogen oxides) and their localization (better than 5-meter accuracy) at a standoff distance of 50 m. Based on the Phase I results, we will define a clear path forward for implementing the DL architecture with low size, weight, and power (SWaP) hardware for edge computing. In Phase II, Intellisense plans to mature the CIGMA design for a field-portable computational platform that can accept and fuse multiple sensor inputs and meet the low SWaP requirements for deployment of the computation platform on small unmanned vehicles (UxV). The Phase II prototype will be tested and evaluated on multiple threat vectors in both laboratory and operationally relevant environments (under assistance from government personnel) to demonstrate improved threat detection and identification over single-sensor DL architectures.