Workflow Guidance using XR and Gesture-Driven User Interaction

Award Information
Agency: National Aeronautics and Space Administration
Branch: N/A
Contract: 80NSSC18P2200
Agency Tracking Number: 181013
Amount: $124,971.00
Phase: Phase I
Program: SBIR
Solicitation Topic Code: Z5
Solicitation Number: SBIR_18_P1
Solicitation Year: 2018
Award Year: 2018
Award Start Date (Proposal Award Date): 2018-07-27
Award End Date (Contract End Date): 2019-02-15
Small Business Information
258 Harvard Street STE 312, Brookline, MA, 02446-2904
DUNS: 080463034
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: N
Principal Investigator
 Salil Patel
 (401) 274-3064
Business Contact
 Sam Karnes
Phone: (401) 345-3320
Research Institution

Access to up-to-date information, with contextual relevance, is critical to advancing the mission objectives for human exploration in space. Crew efficiency and safety depends on the associated workflows.

The proposed project aims to develop a new user experience (UX) for information visualization, with a custom user interface (UI), navigated via micromovement sensors. This innovation allows crew members to access visual data, even while their hands are occupied during a maintenance/support task.

This proprietary control functionality (Pison device) will be combined with commercially-available X-R (Augmented, Hybrid, and Virtual Reality) technology. The Pison device utilizes neuromuscular and Inertial Measurement Unit (IMU) inputs, which are translated into a UI control signal. The UI will be displayed on smart glasses, which allows for navigation of procedures, while visual cues and unobtrusive non-visual notifications are also presented to the user.

This system will be tested in a simulated environment, on the ground, to ensure acceptability for end-users. The proposed proof-of-concept trial will incorporate the display/navigation of step-by-step instructions, response to notifications, and assembly of a physical object. The user will maintain visual access to their surroundings, as well as the procedural instructions. In addition, the system can query the user’s attention through notifications (auditory/haptic), which the user will acknowledge using the Pison device. Subjective and objective measures of task performance and efficiency will be collected in order to refine the hardware design. This will allow for future integration of the technology into standard mission workflow and on-board systems (Phase II).

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government