You are here

Human Computer Interfaces for supervisory control of Multi-mission, Multi-Agent Autonomy


Objective: Develop and demonstrate novel decision support concepts and supporting Human Computer Interfaces (HCI) for the supervisory control of multiple autonomous systems, concurrently engaged in multiple missions. Description: The next generation of unmanned platforms for Navy missions must be capable of autonomously responding to multiple, competing needs that will arise dynamically over the course of a missions. As described in [1], achieving fully autonomous systems will require the system to transact on the basis of goals and translate them into a series of tasks to be performed without extensive human interaction. These tasks may need to be revised over the course of a mission as a function of unanticipated mission events, or emergent mission needs; yet within the constraints established for safe operations. A human will still need remain in the loop in supervisory capacity. However, during an emergency or with an urgent change in mission goals, there may be significant time delays before human intervention occurs. Therefore it may be assumed that these systems have high levels of autonomy that operate through a sophisticated supervisory control model. This topic is intended to develop innovative solutions for the unique requirements of operating highly autonomous systems. This need is critical due to the urgent need to move to a single controller being able to monitor and control multiple unmanned systems and performing across domains as collaborating teams, as documented in numerous Department of Defense objectives [2]. The ability to efficiently managing multiple goals for autonomous systems will be a key enabler to reducing the operator-to-platform ratio. The current state-of-the-art in autonomy mission management technology is limited to relatively few pre-specified goals and very little to no capability to adapt the goals without significant off-board (human) mission planning. In contrast, the next generation of autonomous systems will attend to multiple changing goals arising from a rapidly changing situation on the ground. Methods and algorithms are required to enable an efficient interface of mission goals and tasks for the management of multiple software agents or platforms. This will provide the human planner the flexibly to redirect the platform and reduce the number of specialized operators needed for platform management. In particular, methods are required for mixed-initiative goal formulation, goal prioritization, goal retraction and goal-based agent autonomy. The desired interface research should address a modular architecture that describe: 1) Reasoning approaches for mixed initiative goal management in multiple and concurrent,( but potentially related), missions; 2) An ontology for mission types, tasks, and plans with approaches for integration to the efficient display and manipulation of mission essential tasks; 3) A notional ontology for autonomous platforms capabilities and capacities; and 4) Knowledge authoring tools for autonomy management knowledge bases (e.g., the ontologies themselves). Phase I: Develop an initial design that considers appropriate human factors design principles in providing an intuitive interface for managing mission goals in real-time to enable the user to rapidly adapt to changes in mission context (Note, reports on interfaces for autonomous platforms are generally geared toward the engineer who has great insight into how the systems works, and not the operator who may be focused on meeting mission objective and may not care how the internal system works [3]. This design specification must address functionality from the human supervisor perspective); a workflow that is able to recommend sequences of goals that should be pursued to meet mission requirements; and definition of metrics to assess the improvement in overall system performance against a baseline capability. Phase II: Develop a detailed design and implement an advanced prototype of the autonomy management platform. Demonstrate the effectiveness of autonomy management in multi-mission operations in hostile or adversarial environment and derive and empirically demonstrate improved operator responsiveness to real-time changes in mission execution and/or changes in goals. This phase should involve: specifications for how the technology would integrate with an actual autonomous platform and rationale for selection; rationale for the selection of the appropriate open source software platform; documentation of the Concept of Operations for representative sets of missions; documentation packet for the Institutional Review Board to enable experimentation with Subject Matter Experts; and data collection and analysis techniques to support the evaluation of metrics defined in Phase 1. Lastly, develop a transition plan. Phase III (PRIVATE SECTOR COMMERCIAL POTENTIAL/DUAL USE APPLICATION): Refine the prototype and make the feature set complete in preparation for transition into the Navy. In addition to the Department of Defense, there will be a considerable demand for managing goals associated with autonomous systems in the commercial sector, federal and state agencies such as law enforcement and emergency management. For example, research and development in the field of robotics in the entertainment industry and home service industry can benefit from goal management techniques. At the state level, local police agencies now use lightweight unmanned systems to support surveillance, and as these systems become more complex and gain wide spread use, the ability to manage goals for different law enforcement missions will be required.
US Flag An Official Website of the United States Government