Gamified Analysis Tasks for Heightened Engagement across Repetitions (GATHER)

At the National Geospatial-Intelligence Agency (NGA), the ability to serve and analyze data is crucial to the success of efforts ranging from disaster relief to strategic military support. NGA recently created the Office of Automation, Augmentation, and Artificial Intelligence (AAA) which strives to automate routine tasks to give crucial time back to employees. These automated systems must provide high-confidence results that can be relied on by analysts. To earn this confidence, machine learning algorithms require large training data sets with highly accurate labels. The image analysis process to compile this training data is tedious, not engaging, and often outright boring. To overcome these challenges, under a Phase I effort, Charles River Analytics designed and demonstrated the feasibility of Gamified Analysis Tasks for Heightened Engagement across Repetitions (GATHER). GATHER produces training data for automated image intelligence algorithms using a gamified framework for geospatial intelligence (GEOINT) production and verification. Within this framework, we developed a novel, lightweight survival-crafting game, At the Map’s Edge, which shapes narratives around gamified analysis tasks that motivate analysts to perform high-quality, efficient image content extraction. Based on the success of our Phase I effort, we propose a Phase II effort to develop and evaluate a full-scope prototype of GATHER. Under Phase II, we will extend our Phase I prototype to include: (1) a full-scope Game Environment prototype inspired by popular games, such as Don’t Starve and AdVenture Capitalist, which replaces typically repetitive game mechanics designed to meter out game achievement with gamified tasks that produce useful output; (2) a full-scope Engagement Manager prototype, which calculates optimal configurations of engagement mechanisms to achieve automation training goals, and uses established techniques and measures of analyst engagement to adjust the analyst’s perceived incentives and improve task engagement; (3) the Game Environment and Engagement Manager integrated with existing imagery analysis tools through the development of a prototype Gamified Task Overlay, which provides real-time feedback and information about game progress while an analyst uses a standard web-based analysis tool to produce automation training data. Finally, we will execute a testing and evaluation plan based on early and frequent participatory tests to evaluate engagement mechanisms and overall system usability and refine the prototype systems. The plan culminates in a summative study that compares the output and user attrition of a baseline tool against the game-based approach used by GATHER. This study will demonstrate GATHER’s ability to increase the speed and effectiveness of training data generation, ultimately providing higher quality automation to free analysts for the most challenging and high-impact tasks.