A Smart Interaction Device for Multi-Modal Supervisory Control of Multiple RPAs

Award Information
Agency: Department of Defense
Branch: Air Force
Contract: FA8650-12-C-6305
Agency Tracking Number: F103-036-0983
Amount: $749,946.00
Phase: Phase II
Program: SBIR
Awards Year: 2012
Solicitation Year: 2010
Solicitation Topic Code: AF103-036
Solicitation Number: 2010.3
Small Business Information
3600 Green Court, Suite 600, Ann Arbor, MI, -
DUNS: 009485124
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: N
Principal Investigator
 Glenn Taylor
 Research Scientist
 (734) 887-7620
Business Contact
 Andrew Dallas
Title: Vice President, Federal S
Phone: (734) 887-7603
Email: contracts@soartech.com
Research Institution
ABSTRACT: Demand for Remotely Piloted Aircraft (RPA) within the U.S. Military has significantly increased over the past decade, including Micro-Air Vehicles (MAVs). However, effective use of these systems has been hindered by the lack of natural interfaces for controlling them. Currently multiple operators are required to manage all aspects of a single aircraft, and pilots interact using joysticks or point-and-click interfaces that require their full attention. The work proposed here seeks to enable one operator to exert supervisory control over multiple MAVs using multiple modes of interaction in order to reduce the burden on the operator while maintaining situational awareness. To accomplish this, we will build and evaluate a prototype multi-modal Smart Interaction Device for MAVs (SID/MAV), which will allow pilots to use natural means of interacting such as speech and gesture to interact with multiple MAVs. We will extend our prior work to this new domain, including new input modalities and a new plug-in to interface with RPA control systems via STANAG 4586 messaging. We will evaluate SID/MAV on a portable tablet in a laboratory setting and in a live-flight setting. SID/MAV will enable supervisory control of one or more RPAs, and greatly reduce the workload of the RPA operator. BENEFIT: A natural multi-modal interaction system for supervisory control of RPAs has the potential to change the face of RPA systems. Current battlefield RPAs effectively require multiple people to operate, including navigation, system health monitoring, and payload and mission management. Changing the way in which an operator can interact with each RPA, using more natural input modalities and including feedback from that RPA in natural modalities, frees the operator to work at a higher level of interaction. This in turn will allow for a single operator to more easily manage multiple RPAs, much like a human team lead can manage multiple subordinates. Our system, a Smart Interaction Device for Micro Air Vehicles (SID/MAV) will be capable of managing diverse RPAs though a single natural control interface. SID/MAV will be deployable to multiple computing platforms, such as tablets or laptops, enabling wide access to RPA control, either in a control center or in the field. Beyond RPA applications in reconnaissance, surveillance, and target acquisition (RSTA) missions, supervisory control of autonomous entities has a great deal of potential in a range of fields: managing all kinds of ground and airborne unmanned systems; managing entities in large-scale simulation environments, where hundreds or even thousands of entities play a part; and managing intelligent virtual characters in immersive trainers.

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government