Robust and Sequential Recognition of Emotional States Using Hidden Markov Models and Integrated Speech, Video, and Thermal Features

Award Information
Agency: Department of Defense
Branch: Defense Advanced Research Projects Agency
Contract: W31P4Q-04-C-R150
Agency Tracking Number: 03SB2-0270
Amount: $98,815.00
Phase: Phase I
Program: SBIR
Awards Year: 2004
Solicitation Year: 2003
Solicitation Topic Code: SB032-038
Solicitation Number: 2003.2
Small Business Information
LI CREATIVE TECHNOLOGIES
225 Runnymede Parkway, New Providence, NJ, 07974
DUNS: 125624432
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: Y
Principal Investigator
 Qi Li
 President
 (908) 508-0239
 qili@ieee.org
Business Contact
 Qi Li
Title: President
Phone: (908) 508-0239
Email: qili@ieee.org
Research Institution
N/A
Abstract
This proposal defines unique and promising solutions for robust and sequential emotional and stress state recognition in military operational environments. The solutions are non-invasive, multi-modality approaches, including speech, video, and thermal images. The feature sets are selected automatically based on the operational environments. For example, when the environment is too noisy, the system focuses on image features; when the lighting condition is too bad, it focuses on speech features; when both acoustic and lighting conditions are good, the system uses both speech and image features. This will provide the military with a degree of system versatility and allow broad applications to many operational scenarios. Multi-layer hidden Markov models are defined as the statistical models to characterize the emotional states. The recently developed sequential detection algorithm is proposed to detect the changes from one emotional state to another. Furthermore, a multi-modality database will be collected to study the feasibility of detecting at least the following emotional states in this project: anger, drowsiness, anxiety, fear, confusion, disorientation, and frustration. The new feature extraction and recognition algorithms will be developed and implemented through this project to discriminatively recognize those states. The developed system will be portable and can be operated in military environments automatically and continuously.

* information listed above is at the time of submission.

Agency Micro-sites

US Flag An Official Website of the United States Government