You are here

Glove-based Tactile Streaming of Braille Characters and Digital Images for the Visually Impaired

Award Information
Agency: Department of Health and Human Services
Branch: National Institutes of Health
Contract: 1R41EY034738-01
Agency Tracking Number: R41EY034738
Amount: $257,200.00
Phase: Phase I
Program: STTR
Solicitation Topic Code: NEI
Solicitation Number: PA21-262
Timeline
Solicitation Year: 2021
Award Year: 2023
Award Start Date (Proposal Award Date): 2023-09-30
Award End Date (Contract End Date): 2024-09-29
Small Business Information
2701 MCMILLAN AVE STE 160
San Luis Obispo, CA 93401-4744
United States
DUNS: 080932258
HUBZone Owned: No
Woman Owned: No
Socially and Economically Disadvantaged: Yes
Principal Investigator
 MICHAEL EICHERMUELLER
 (805) 888-4275
 michael@haptx.com
Business Contact
 KIMBERLY OBERG
Phone: (805) 888-4278
Email: kim.oberg@haptx.com
Research Institution
 OLD DOMINION UNIVERSITY
 
ODU RESEARCH FOUNDATION 4111 Monarch Way, Suite 204
NORFOLK, VA 23508-0369
United States

 Nonprofit College or University
Abstract

This project will achieve a computerized system that supplies digital content, both lexical and graphical,
suitable for children, students and working adults of the Visually Impaired (VId) community, to increase their
autonomy and potentiate their Quality of Life and societal contribution. While there is notable academic
research, such as a high-density pin display tablet, there is no commercial, broadly functional, full-page
Braille reader. A tablet reader is potentially limiting in comparison with bimanual haptic gloves, limiting any
content complementary to Braille text. In contrast, haptic gloves support both Braille lexical content and
palpable digital images.The proposed system will encapsulate a full-length Braille reader, not limited to a page at a time but
supplying a limitless stream of characters readable by four fingers in each hand as dictated by the content,
while also supporting haptic actuation of embedded pictorial content. This glove-based approach will unlock,
for an underserved VId clientele, a vast collection of multimedia content currently the exclusive domain of
the seeing population, as Braille lexical content and palpable bas-relief of graphical content.Phase I will center on a haptic system for Braille display of full-length text, sensed by the fingers of both
hands and based on a redesigned commercial bimanual glove-based haptic system. A software algorithm
will stream and stagger the Braille content to each finger to replicate the bimanual reading process of a VId
reader. The system will also feature a tactile bas-relief version of digital images embedded in the text.Phase II will develop Haptic VR, demonstrating the utility of overlaying Braille characters on a haptic
curved surface and paving the way for vitally important haptic sensing of labeled 2D/3D graphics by VId
users. A long-term goal is to apply this VR foundation to geolocated Haptic AR, enabling haptic overlay of
Braille labels on everyday surfaces, from appliances to groceries, by suitable integration of haptic rendering
with 3D range-sensing and DNN-based object recognition. Specific Aims – The Phase I Aims are as follows.Aim 1. High-resolution haptic gloves for Braille streaming- Development of bimanual high-resolution haptic
gloves, featuring dense actuation at eight fingertips, by Haptx. The thumbs will also get haptic feedback.Aim 2. Haptic rendering of lexical and graphical content2A. Lexical reader - A software transcription will support spatiotemporal streaming of lexical data that
moves the haptic footprint of Braille characters serially from the first to the eighth finger.2B. Graphical reader – The conversion of a digital image to a haptic bas-relief will exploit fast DNN-based
image semantic segmentation, where each label will be mapped to an actuator pressure.2C. Finger-based calibration- Both modalities will be supported by a virtual page defined on a tabletop,
whose coordinates are found by a simple calibration based on three points identified by the index fingers.

* Information listed above is at the time of submission. *

US Flag An Official Website of the United States Government