A New Approach to the Testing & Evaluation of Advanced RF Applications of Deep Learning AI

The major advances in artificial intelligence (AI) in the past ~10 years have been the result of the progress in so-called deep learning (neural) networks (DLNs). Moreover, several recent Air Force (AF) projects have demonstrated the efficacy of DLNs for very advanced radio frequency (RF) applications of direct relevance to some of the most critical AF missions including Intelligence, Reconnaissance, & Surveillance (ISR), and Suppression of Enemy Air Defenses (SEAD) against near-peer Integrated Air Defense Systems (IADS). RF sensing applications that have shown significant progress include advanced air-borne/spaceborne ISR radar (look-down), Electronic Intelligence (ELINT), and Radar Warning Receivers (RWR). However, to ultimately transition these technologies to operational status, Test & Evaluation (T&E) methods must be developed that meet the rigorous reliability and sustainability goals that have long been established by the Department of Defense (DoD) T&E community. A fundamental obstacle to the direct adoption of DLN based AI solutions is the relatively “black box” nature of DLNs. “Understanding” how a DLN is making decisions is the focus of so-called Explainable AI (XAI) research. Even if some measure of understanding is achieved, to meet accepted criteria for DoD weapon system availability (referred to as Ao, pronounced “A-sub-O”), rigorous statistical evidence must also be provided that establishes the statistical reliability of the new system. In this proposal, a new comprehensive and end-to-end (E2E) approach to designing, developing, testing, and validating DLN AI solutions for advanced RF applications is presented. At its core is an advanced RF Digital Engineering (DE) engine built on the highly successful RFView™ family of DE products: Mod & Sim, Hardware-in-the-Loop (HIL), etc. The high-fidelity, physics-based, and vetted RF solver (VHF through Ku bands) enables both voluminous synthetic data, and a virtual reality (VR) environment to overcome the fundamental “data starved” limitations of real-world military systems attempting to utilize DLN AI methods. This new approach allows for unprecedented levels of virtual (and augmented “live-over-sim”) flight tests both purely digital and hybrid via the HIL options. Thus, even real world and real-time hardware/software issues can be thoroughly fleshed out without prohibitively expensive flight testing. This new approach also allows for extensive “excursion-from-baseline” analysis to ensure the final product is robust to potentially unforeseen operating conditions. This kind of excursion analysis is simply too expensive and impractical to be conducted via actual flight tests. The development of robust excursion test vectors can be assisted with other AI techniques. Lastly, but critically, this DE environment allows for the generation of sufficient test vectors to achieve any prescribed level of statistical reliability as required to establish a given weapon system availability, Ao