SBIR Phase II: Virtual and Augmented Reality Enabled Personalized Manufacturing Training

The broader impact of this Small Business Innovation Research SBIR Phase II project will be achieved through investigating approaches to leverage extended reality (XR) for immersive, interactive and intelligent training in manufacturing, healthcare and higher education, to accelerate learning, increase engagement, and foster more effective use of expensive equipment. In 2017, the global training market was estimated to be $388 B, while the US market was estimated to be $166 B. Training and design use cases frequently rank among the top 5 XR use cases in industry reports, and the enterprise VR training market alone is forecasted to grow at 140% year over year for the next five years to reach $6.3 B per year. XR technologies hold significant potential by providing users with a sense of presence and immersion heretofore not possible due to cost, safety, security, physical, geographical or other limitations. This project will pursue this growing market opportunity with content expected to bring together classrooms, labs, factories, hospitals and field service environments, broadly advancing learning and engagement in both schools and workplaces. The intellectual merit of this SBIR Phase II project lies in the development of high- fidelity modular and reusable virtual object and assembly models, customized models for scenarios, tasks, instructions and user proficiency, application of natural language processing techniques to enable conversational interactions in immersive multiplayer environments and the use of computer vision techniques to support effective reuse of content across both virtual and augmented reality settings. Specialized modules will be developed for each of the above functionalities and then integrated to develop an innovative, easy-to-use product that will make it easy for non-programmers to author and publish XR content for training, design, service and other use cases, as well as collect data to evaluate the content and experiences. Pilot testing will be conducted with users in the higher education industry to rigorously evaluate and iteratively refine the technologies and the final product. By judiciously combining XR, virtual objects, and simulations with voice, vision and collaboration, this project aims to build an innovative product that may enable anyone to build engaging knowledge-rich content. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.