Training-Assessment-Feedback Loop to Empower the Information Warfighter

The information environment has become a battlefield for adversaries of the United States and its allies. Open-source intelligence analysts need training to learn effective methods to counter adversarial information operations, tools to carry out the methods, an opportunity to practice the methods in a realistic environment, and near real-time feedback to hone mission-critical skills. The proposed system (FLOW, a feedback loop for operational workflows) is a web-based application that operates in parallel with the operational social media data analytics tool Scraawl (www.scraawl.com) to support both training and operations. FLOW offers instructional courses with interactive lessons to teach analysts important information-operation workflows. Using carefully curated social media datasets in Scraawl, analysts explore and analyze the information environment to identify, track, and evaluate bot-driven information campaigns. Analysts get the opportunity to practice their skills in a realistic training environment instrumented to collect traces of their workflows and provide contextual feedback on the quality of their actions. FLOW evaluates performance using the analysts’ actions in Scraawl and their responses to prompts in the instructional lessons. As lessons advance, instructional scaffolding is gradually removed so that analysts begin performing more independently. Flow also supports analysts during operational analyses. FLOW will capture workflow traces and allow analysts to document their reasoning at critical decision points. The workflow traces will serve as analytic artifacts (for report deliverables) and allow analysts to compare their workflows to those of experts as a way to get feedback and continue improving their analytic skills. Finally, FLOW provides interactive workflow templates to guide analysts in conducting sophisticated analytic workflows while also documenting the process so that they can report their findings with adequate justification.