Agile Machine Learning in Dynamic Environments for Complex Event Processing

RT&L FOCUS AREA(S): Artificial Intelligence/Machine Learning

TECHNOLOGY AREA(S): Information Systems

OBJECTIVE: Develop a system to detect and classify complex events that can be quickly retrained for different operating environments using sparse training data by leveraging human domain expertise.

DESCRIPTION: Effective network command, control, communications and intelligence in MDO environments requires situational understanding across multiple domains in order to achieve overmatch against the adversary. In the fight against a peer adversary, it is anticipated that such overmatch will require short decision cycles in the order of minutes or less. To this end, fully autonomous solutions are needed to ingest sensor data and contextual information across the battlespace to provide actionable recommendations to the decision maker. Today, automated solutions for target detection and classification are beginning to appear to be possible across the cyber and various physical (land, air, sea, space) domains. However, intelligence analysts would still be needed to interpret these warnings and indicators to correlate the observed movements of various entities across domains into coordinated adversarial tactics, techniques, and procedures (TTPs). Such coordination represents a sequence of atomic activities across fairly large scales of time and space and broadly speaking fall under the guise of complex events in the data science literature [1]. For instance, the amalgamation of many coordinated atomic loitering and movement activities of vehicles and dismounts can represent a single red force reconnaissance complex event. It is desirable for automated systems to report as many true complex events as possible while limiting the number of unique false events reported per an extended interval of time, e.g., hours.

Deep learning methods have successfully classified simple activities that span short time intervals and are localized such as sports and daily activities [2-3]. However, deep learning has yet to provide satisfactory solutions for complex event processing [4]. Furthermore, the rapidly changing and unpredictable MDO environment means that limited data will be available for training, and thus, it is expected that a purely end-to-end deep learning solution would be inadequate. Recently, it has been demonstrated that human domain expertise can be combined with deep learning to recognize complex events [5] and that neuro-symbolic machine learning can adapt data-driven machine learning to recognize atomic concepts without explicit labels using limited data with only the complex events labeled [6-7]. While these results are promising, the complex event processing methods have yet to be developed and evaluated for recognition of existing and emerging TTPs in the MDO setting.

The goal of this topic is to advance and demonstrate the state of complex event processing systems to enable reliable classification over a large set of possible TTPs that span over various domains. To this end the performer will need (1) define a set of relevant complex events that can capture the richness and variability of possible workflows for each event, (2) collect data for such complex events across multiple domains in a variety of operational environments (e.g., weather conditions, operational tempos, etc.), (3) develop complex event processing solutions that incorporate both domain expertise and data-driven learning, and (4) demonstrate classification performance over the complex event processing solution. For the purposes of feasibility, it is expected that the data collection will fully leverage synthetic data generation technologies for at least phases I and II.

PHASE I: Define at least three complex events that entail data collected from one domain. Each complex event must entail at least three atomic activities separated in time by the order of minutes and in space such that the field of view of a single sensor is unable to detect the event. The data collection should represent at least two different operating environments and contain background atomic activities that are not correlated to any of the three complex events. The complex event classification systems should be trained over large amounts of data for one operating condition, and then adapted for the second operating condition using a significantly smaller fraction of training data. Overall, the complex event classification system should exhibit a probability of detection of at least 0.9 with a false report rate of one per hour. Furthermore, the probability of correct classification should exceed 0.9 for the full trained operating condition while still achieving a probability of correct classification of 0.8 for the adapted operating condition.

PHASE II: Define at least ten complex events that entail data collected from two or more domains. Each complex event must entail at least ten atomic activities and at least one complex event must exhibit a timespan of one hour. Furthermore, five of the complex events must be separated in space such that more than three spatially disparate sensors are required to detect these events. The data collection should represent at least three different operating environments and contain background atomic activities that are not correlated to any of the three complex events, but much of the background activities must be correlated to normal civilian activities such as commuting to work, running errands, etc. The complex event classification systems should be trained over large amounts of data for one operating condition, and then adapted for the two other operating condition using a significantly smaller fraction of training data. Overall, the complex event classification system should exhibit a probability of detection of at least 0.9 with a false report rate of one per hour. Furthermore, the probability of correct classification should exceed 0.9 for the fully trained operating condition while still achieving a probability of correct classification of 0.8 for the adapted operating conditions.

PHASE III DUAL USE APPLICATIONS: The complex event processing system should be able to allow domain experts to quickly define adversarial TTPs across any domain and the system should be able to train over modest amounts of labeled data. Such technology should benefit the civilian sector by enabling law enforcement to detect and classify emerging complex events by foreign and domestic terrorist groups using open source signals collected over social media as well as monitoring attacks of various infrastructure systems in the cyber domain.

REFERENCES:

1. Flouris, I., Giatrakos, N., Deligiannakis, A., Garofalakis, M., Kamp, M., & Mock, M. "Issues in complex event processing: Status and prospects in the big data era." Journal of Systems and Software 127 (2017): 217-236.
2. Wu, Z., Wang, X., Jiang, Y. G., Ye, H., & Xue, X. "Modeling spatial-temporal clues in a hybrid deep learning framework for video classification." In Proceedings of the 23rd ACM international conference on Multimedia, pp. 461-470. 2015.
3. Herath, S., Harandi, M., & Porikli, F.. "Going deeper into action recognition: A survey." Image and vision computing 60 (2017): 4-21.
4. Alevizos, E., Skarlatidis, A., Artikis, A., & Paliouras, G. "Probabilistic complex event recognition: A survey." ACM Computing Surveys (CSUR) 50, no. 5 (2017): 1-31.
5. Liu, X., Ghosh, P., Ulutan, O., Manjunath, B. S., Chan, K., & Govindan, R. "Caesar: cross-camera complex activity recognition." In Proceedings of the 17th Conference on Embedded Networked Sensor Systems, pp. 232-244. 2019.
6. Xing, T., Garcia, L., Vilamala, M. R., Cerutti, F., Kaplan, L., Preece, A., & Srivastava, M. "Neuroplex: Learning to detect complex events in sensor networks through knowledge injection." In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pp. 489-502. 2020.
7. Vilamala M.R., Xing T., Taylor H., Garcia L., Srivastava M., Kaplan L. Preece A., Kimmig A., Cerutti F. Using DeepProbLog to perform Complex Event Processing on an Audio Stream. In Proceedings of the Tenth International Workshop on Statistical Relational AI (https://arxiv.org/abs/2110.08090). 2021.

KEYWORDS: artificial intelligence, machine learning, complex event processing, situational understanding, classification, detection