You are here

Deep Reinforcement Learning (DRL) Enabled Warfighter Assistant

Description:

OUSD (R&E) MODERNIZATION PRIORITY: Network Command, Control and Communications; Autonomy; Artificial Intelligence/Machine Learning; General Warfighting Requirements

 

TECHNOLOGY AREA(S): Weapons; Sensors; Information Systems; Human Systems; Battlespace

 

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

 

OBJECTIVE: Create explainability tools and methods to be applied to machine learning (ML) agents capable of assisting warfighters in training and operational scenarios.

 

DESCRIPTION: With the growing complexity of the world, the warfighter’s decision space has become enormous.  While there are existing techniques, tactics, and procedures defined by an engagement doctrine, it is infeasible for a human to predict and respond to the entire set of intricate strategies an adversary might present.  Recent advances in DRL have been able to create powerful agents capable of making decisions under uncertain and complicated scenarios, but the reasoning behind the DRL suggestions is often opaque.  Humans must remain in the loop during operations due to the high-consequence nature of the environment and they must also have confidence that the DRL recommendation is trustworthy.  Therefore, in addition to developing these ML capabilities, there is a need to also transition them into assistive tools for humans.  This topic seeks to (1) develop explainable reasoning behind DRL generated recommendations and (2) integrate them into intuitive and interpretable tools capable of assisting analysts and decision makers.  This would allow for increased adoption and trust of ML based solutions and enhanced warfighter effectiveness in assessing current status of forces, threats, and response options.  Courses of action can be recommended using a DRL trained agent and explained with assistive tools.

 

PHASE I: Develop proof-of-concept algorithms, tools, software and analyses that demonstrate potential for achieving the topic objectives:

- Identify notional simulation and scenario for DRL

- Develop explainability features and increase transparency of DRL agent decision making

- Explain factors influencing DRL based suggestions and actions

- Develop concepts for explainability and trust tool integration into analyst and/or warfighter interfaces

 

PHASE II: Develop a full prototype capability demonstrating initial capabilities per topic objectives (per Phase I) with the intent of testing the capability for experimentation in government Modeling and Simulation labs using government provided DRL framework.  This should include prototype level user and design documentation.  Development should facilitate cyber security approval for loading the prototype software on government computer systems through cyber aware design decisions and development of cyber security artifacts.

 

PHASE III DUAL USE APPLICATIONS: Develop operational capability for use in government simulations, including user and design documentation. Maintain and improve capabilities based on Phase I and Phase II use experience.  Continue to support cyber assurance.

 

REFERENCES:

  1. https://deepmind.com/research/case-studies/alphago-the-story-so-far 
  2. https://www.darpa.mil/news-events/2020-08-07  Advanced algorithms to fly simulated F-16 dogfights against each other, Air Force pilot in online finale

 

KEYWORDS: Deep Reinforcement Learning; Data Driven Models; machine learning; agent based models; decision making; decision aids; recommendation engines; operator assistants

US Flag An Official Website of the United States Government