STTR Phase II: Dynamic Robust Hand Model for Gesture Intent Recognition

The broader impact/commercial potential of this project stems from addressing the important hand gesture based input challenges of VR/AR (expected to grow to $150B by 2020), Robotics and IoT ($135B by 2019 and $1.7T by 2020 respectively). This technology, if successful in mitigating the high technical risks, represents a huge leap in the state of the art in 3D hand models for gesture recognition and has the potential to be the industry standard for AR, VR, Robotics and IoT applications with broad societal impact in education, medical and healthcare. Its broader impact is further amplified by the potential in serving the needs of the disabled community in improving their quality of life by being better able to communicate, learn and adapt to their interaction needs. This Small Business Technology Transfer (STTR) Phase 2 project aims to significantly advance current 3D hand gesture recognition technology by developing a dynamic hand tracking model for gesture intent recognition. It is robust against occlusion and tolerant to variations in camera orientation and position. This research will result in a transformative leap above the current state of academic and commercial hand models and overcome key technical hurdles that have so far proven difficult to overcome. It solves the following key challenges and involves very high technical risks: 1) robust hand tracking while holding objects and 2) robust tangible interactions using objects without using any fiducial markers 2) low profile hand wearable for touch interaction detection. This Phase 2 project will achieve these objectives by 1) data acquisition and hand-object pose estimation, 2) understanding user intents with enhanced tangible interactions, and 3) system validation and user testing.