You are here

Workable Hierarchical Impersonation using Reinforcement Learning (WHIRL)

Award Information
Agency: Department of Defense
Branch: Defense Advanced Research Projects Agency
Contract: HR001123C0118
Agency Tracking Number: D2D-0469
Amount: $1,791,604.00
Phase: Phase II
Program: SBIR
Solicitation Topic Code: HR0011SB20234-02
Solicitation Number: 23.4
Timeline
Solicitation Year: 2023
Award Year: 2023
Award Start Date (Proposal Award Date): 2023-08-14
Award End Date (Contract End Date): 2026-08-14
Small Business Information
4075 Wilson Blvd, STE 800
Arlington, VA 22203-1798
United States
DUNS: 080709338
HUBZone Owned: No
Woman Owned: No
Socially and Economically Disadvantaged: No
Principal Investigator
 Brian Weigel
 (202) 838-6006
 brian.weigel@cynnovative.com
Business Contact
 Matt Puglisi
Phone: (410) 533-3817
Email: admin@cynnovative.com
Research Institution
N/A
Abstract

Workable Hierarchical Impersonation using Reinforcement Learning (WHIRL) will generate realistic synthetic data without artifacts at scale by utilizing hierarchical reinforcement learning and a hypervisor to allow for “off-box” execution of long-term goals, mid-term tasks, passed through a shim to a hypervisor that will execute them on the intended host. Team Cynnovative will use hierarchical reinforcement learning to simulate user behavior at the level a real user would: keyboard and mouse activity and observing a monitor. By simulating on real hardware and executing “off-box,” WHIRL enables the collection of the generated synthetic data via any traditional means a WHIRL user desires without introducing any artifacts or biases. User persona research for SUP is a method of understanding the characteristics and behavior patterns of specific groups of users to gain insights into the motivations, goals, and needs of these user groups and inform the design and development of effective cybersecurity strategies. This research seeks to understand how users interact with network systems, applications, and data to design policies that enable a user to operate successfully while maintaining a robust security posture. The autonomous agent for WHIRL is rewarded for taking actions to achieve a goal, such as browsing the web or using an excel sheet or operating in a terminal. Feedback from the environment informs the agent how well it accomplishes the task. A fully trained agent can act as a defined synthetic user enables the generation and collection of robust datasets representative of realistic user behavior. Team Cynnovative’s solution will be operating on the raw pixels of a screen capture which puts reinforcement learning in a real-world domain with an observation space where it has succeeded in the past, effectively eliminating the simulation to real-world problems. Reinforcement learning can operate on pixel data and elicit realistic, emergent behaviors with ground truth. WHIRL operates “off-box” on real hardware (via hypervisor), enabling the collection of synthetic data the way the data is normally collected. This means that the users of the WHIRL system will not have to worry about learning how to collect via another platform but will enable them to leverage existing knowledge and tools and not worry about filtering for any artifacts or biases.

* Information listed above is at the time of submission. *

US Flag An Official Website of the United States Government