You are here

Using Cognitive Digital Twin Framework for Autonomous Target Discrimination


OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy; Advanced Computing and Software; Integrated Sensing and Cyber; Emerging Threat Reduction


The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.


OBJECTIVE: Seek the Cognitive Digital Twin technology to develop a novel method for fusion of heterogeneous multi-sensor information to characterize and understand environment, identify high interest objects, and support intelligent decision-making.


DESCRIPTION: To protect the United States and allies from current ballistic missiles and preparing to defend against future increasingly more complex threats, the BMDS assigned the high priority to advance current architecture models to incorporate novel methodologies and techniques.  This architecture model advancement can be also performed in the commercial domain and the obtained results transferred to military applications.


One of the most critical characteristics of many military and commercial assets is its ability to utilize all the available information from deployed sensors and the state-of-the art Machine Learning (ML) and Artificial Intelligence (AI) algorithms to perform their important functions as scene characterization and understanding, target selection and decision-making.  To boost quality of these tasks, an autonomous system needs to be designed and developed that employs an information fusion scheme with contextual adaptation, a time-evolving ML/AI discrimination of high priority objects, and fire engagement decision-making procedures derived from estimated classification confidence.


Current machine learning approaches to information fusion and decision-making in both military and commercial often have no means of properly handling differences in sensor resolution and coverage, disparate phenomenology, and limited viewing geometry, as well as the inherent uncertainty of the collection of observed data or/and of exploiting the available higher level contextual information on the threat evolution and environment.


Due to the development of new-generation information and digitalization technologies, more data can be collected, which in turn require better ways for the deep application of these data.  As a result, the concept of Digital Twin (DT) has aroused much attention and is developing rapidly.  DT is typically described as consisting of a physical system, its virtual replica, and the data connections between them.  It is increasingly being explored as a means of describing and improving the performance of physical systems through leveraging various computational techniques including ML and AI algorithms.  The advantage of DT over simulations is that it creates an active virtual environment capable of involving several simulations, utilizing real-time data and a two-way flow of information between the twin and the data sensors.  The DT technology becoming increasingly prevalent in many fields, including the autonomous systems industry.


Characterizing and identification of incurring military or commercial objects in complex environments using diverse sensing has a number of significant challenges.  Typically, data may be sparse, collection times are limited, a priori information is incomplete and/or may be in error, additionally, unexpected events and objects may appear.  Though DT technology can support many of the necessary integration and correlation procedures, it needs to fuse all available data with information and knowledge related to the scene characterization and target identification.  Hence, it becomes necessary to augment the DT with cognitive capabilities.  Semantic technologies such as ontology and knowledge graphs could provide potential solutions by empowering DTs with reasoning abilities.


The Cognitive Digital Twin (CDT) concept has been recently proposed which reveals a promising evolution of the current DT concept towards a more intelligent, comprehensive and full lifecycle representation of complex systems.  It is intended to harness a high level of intelligence that can replicate human cognitive processes and execute conscious actions autonomously, with minimal or no human intervention.  The CDT structure provides an excellent framework to greatly improve the effectiveness of autonomous system actions.  The novel fusion and decision-making system structured as a cognitive digital twin can be suitably integrated with relevant digital twins developed by many developers via its capabilities of communication, analytics, and cognition.  Consequently, it will perfectly support development and evolution of capabilities for autonomous systems.


A CDT based fusion system would replace and/or improve legacy and current approaches like Dempster-Schafer theory, rule-based expert systems, Bayesian networks, probabilistic relational models, etc.

In summary, the successful proposal will address all the technical challenges in designing and developing an onboard CDT based fusion and decision-making system including:

1. Appropriate handling differences in sensor resolution and coverage, disparate phenomenology, and limited viewing geometry, as well as the inherent uncertainty of the collection of observed data;

2. Exploiting the available higher level contextual information on the threat evolution and scenery;

3. Optimizing real-time data collection and minimizing the required data transmission;

4. A time-evolving, adaptive ML/AI important entities identification algorithms providing class probabilities of high priority items as well as dealing with absence of information on some objects;

5. Commitment decision-making procedures derived from estimated classification confidence and contextual reasoning;

6. Supporting future evolution of capabilities of autonomous systems in complex and evolving environments.


PHASE I: Demonstrate proof of principle with a cognitive digital twin prototype for an innovative fusion, classification, and decision-making concept.  Utilizing surrogate objects generation, their data, and contextual information, conceptualize, develop, and model near real-time solutions that satisfy the problem objectives and requirements.


PHASE II: Using realistic, relevant threat data, refine and implement designs from Phase I.  Validate concept with available test data.


PHASE III DUAL USE APPLICATIONS: The topic has numerous military and commercial applications, where fusion of available time-evolving information can assist with scene description and its understanding to support optimal decision-making, e.g., advanced assistance systems and autonomous vehicle development.



  1. Li, Luning, et al. "Digital twin in aerospace industry: A gentle introduction." IEEE Access 10 (2021): 9543-9562.
  2. Hossain, S M Mostaq, et al. “A New Era of Mobility: Exploring Digital Twin Applications in Autonomous Vehicular Systems.” 2023 IEEE World AI IoT Congress IEEE, (2023): 493-499.
  3. Eirinakis, Pavlos, et al. "Enhancing cognition for digital twins." 2020 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) IEEE, (2020): 1-7.
  4. Sahlab, Nada, et al. "Extending the Intelligent Digital Twin with a context modeling service: A decision support use case." Procedia CIRP 107 (2022): 463-468.
  5. Maurer, Donald E., et al. "Sensor fusion architectures for ballistic missile defense." Johns Hopkins APL technical digest 27.1 (2006): 19-31.
  6. Domingos, Pedro and Richardson, Matthew. "Markov logic: A unifying framework for statistical relational learning." Statistical Relational Learning (2007): 339-344.


KEYWORDS: Target Discrimination; Remote Sensing; Information Fusion; Digital Twin; Cognitive Process; Machine Learning; Artificial Intelligence; Autonomous System; Decision-making

US Flag An Official Website of the United States Government