Dynamic 3D Human Shape Modeling for Intention Prediction from Video Imagery

Award Information
Department of Defense
Air Force
Award Year:
Phase II
Agency Tracking Number:
Solicitation Year:
Solicitation Topic Code:
AF 08-030
Solicitation Number:
Small Business Information
11600 Sunrise Valley Drive, Suite # 290, Reston, VA, 20191
Hubzone Owned:
Minority Owned:
Woman Owned:
Principal Investigator:
Atul Kanaujia
Principal Investigator
(703) 654-9300
Business Contact:
Paul Brewer
VP, New Technololgy
(703) 654-9314
Research Institution:
3D human behavior and shape analysis from monocular video imagery is an enabling technology for recognizing abnormal behavior and threat monitoring in military and law enforcement applications. The problem is challenging owing to (i) complexity of surveillance environment, (ii) high dimensionality of human pose space; (ii) lack of depth information; and (iii) large variability in human body shape, clothing and imaging conditions. We propose a robust framework that combines discriminative and generative approaches for inferring 3D pose and anthropometric characteristics of a person. In order to deal with loss of depth information and resolve ambiguities, we use a combination of techniques based on learned dynamical priors, biomechanical characterization of human pose and multi-hypothesis tracking. Each of the techniques aims to constrain the pose search to find the most optimal pose and shape that best describes the person in the image. The system is fully automatic and has modular architecture to support extensibility and facilitate transition to operational deployment. Preliminary results in Phase I validate the feasibility of this approach. Phase II work will involve an effort to refine and optimize the framework to achieve near real-time processing and to enhance the system robustness to support complex poses and varied environments. BENEFIT: The proposed work addresses the technical challenges of human pose and shape estimation in video imagery. It will enable us to detect human activities and to identify suspicious and hostile human behaviors for military and law enforcement applications. The technology has wide ranging applications beyond intelligent surveillance system, including: a) Improved human computer interaction - The framework will facilitate development of more accurate vision-based systems to recognize different gestures and motion in 3D. This has vast potential use in role-playing games where the movements of the user in the physical domain are appropriately reflected as an action in the virtual environment. b) Movement analysis - For identifying the underlying causes for walking abnormalities in clinical patients. The results of gait analysis have been shown to be useful in determining the best course of treatment for these patients. c) Intelligent training systems for sporting activities - The analysis of sports-related movements often entails analyzing a variety of highly dynamic movements. Motion analysis provides the tools for the sports medicine and performance professionals to perform accurate functional evaluations/analyses for clinical and research-oriented purposes. d) Realistic animation - Cost-effective solution to importing realistic body movements in animated characters in videos. Human gait modeling can be used to simulate realistic walking styles. e) Robotic locomotion - Design of robot appendages and control mechanisms to allow robots to move fluidly and efficiently.

* information listed above is at the time of submission.

Agency Micro-sites

SBA logo

Department of Agriculture logo

Department of Commerce logo

Department of Defense logo

Department of Education logo

Department of Energy logo

Department of Health and Human Services logo

Department of Homeland Security logo

Department of Transportation logo

Enviromental Protection Agency logo

National Aeronautics and Space Administration logo

National Science Foundation logo
US Flag An Official Website of the United States Government