You are here

Real-time Scene Labeling and Passive Obstacle Avoidance in Infrared Video

Description:

TECHNOLOGY AREA(S): Electronics 

OBJECTIVE: To develop and demonstrate techniques for labeling frames of infrared video in real time and using those labels and other information to identify obstacles and threats. 

DESCRIPTION: Great progress has been made in automated identification of targets (objects of interest) in single frame infrared imagery. However, less success has been achieved in multi-class characterization of entire images (scene labeling)--with difficulties presented both in the correct classification of many categories of objects and in the computational time needed to process an entire image. Real time capability is essential for obstacle avoidance, threat detection, and navigation in moving vehicles. What is needed a set of algorithms which exploit spatial and temporal context for the computationally efficient scene labeling of video sequences--which will enable the military operator to respond to avoid obstacles and threats in real-time. The problem of threat detection and obstacle avoidance in full motion passive infrared (IR) video is of critical interest to the Army. Vehicle drivers and sensor operators are inundated with many terabytes of video. Human operators are subject to fatigue, boredom, and information overload. To maintain necessary situational awareness, it is vital to automate the video understanding process as much as possible. The problem presents immense computational complexity and is unsolved. Novel deep learning methods have been developed that promise a qualitative breakthrough in machine learning and aided target recognition (AITR) for object detection and classification in video. The approach in this effort should expand these successes to include full motion video understanding and threat detection. 

PHASE I: Show proof of concept for scene labelling algorithms for obstacle avoidance, navigation, and threat detection in full motion IR video. Show proof of concept for algorithms to greatly increase threat classification effectiveness (high probability of correct classification with minimal false alarms). Integrate algorithms into comprehensive algorithm suite. Test algorithms on existing data. Demonstrate feasibility of technique in infrared (IR) video sequences. Distribute demonstration code to Government for independent verification. Successful testing at the end of Phase 1 must show a level of algorithmic achievement such that potential Phase 2 development demands few fundamental breakthroughs but would be a natural continuation and development of Phase 1 activity. 

PHASE II: Complete primary algorithmic development. Complete implementation of algorithms. Test completed algorithms on government controlled data. System must achieve 90% classification rate with less than 5% false alarms. Principle deliverables are the algorithms. Documented algorithms will be fully deliverable to government in order to demonstrate and further test system capability. Successful testing at end of Phase 2 must show level of algorithmic achievement such that potential Phase 3 algorithmic development demands no major breakthroughs but would be a natural continuation and development of Phase 2 activity. 

PHASE III: Complete final algorithmic development. Complete final software system implementation of algorithms. Test completed algorithms on government controlled data. System must achieve 90% classification rate with less than 5% false alarms. Documented algorithms (along with system software) will be fully deliverable to government in order to demonstrate and further test system capability. Applications of the system will be in NVESD Multi-Function Display Program and vehicle navigation packages. Civilian applications will be in crowd monitoring, navigation aids, and self-driving cars 

REFERENCES: 

1: Farabet, C., Couprie, C., Najman, L., and LeCun, Y., "Learning Hierarchical Features for Scene Labeling", IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 35 Issue 8, August 2013, pp. 1915-1929

2:  Albalooshi, F. and Asari, V.K., "A Self-Organizing Lattice Boltzmann Active Contour (SOLBAC) Approach For Fast And Robust Object Region Segmentation," Proceedings IEEE International Conference on Image Processing - ICIP 2015, pp. 1329-1333, Quebec City, Canada, 27-30 September 2015.

3:  I-Hong Jhuo

4:  Lee, D.T., "Video Event Detection via Multi-modality Deep Learning," Pattern Recognition (ICPR), 2014 22nd International Conference on, pp.666,671, 24-28 Aug. 2014

KEYWORDS: Aided Target Recognition, Deep Learning, Neural Networks, Scene Labeling, Threat Detection 

CONTACT(S): 

Mr. James Bonick 

(703) 704-1829 

james.bonick@us.army.mil 

US Flag An Official Website of the United States Government