Deep Transfer Learning Across Domains, Modalities and Classes

The capability of “transferring” learned classifiers from one domain, or set of targets, to classify different targets or the same targets but in different domains is of great interest to the United States Air Force. It is an enabling technology for the USAF to build Aided Target Recognition and other algorithms for environments and targets where the data or labeled data is scarce. In this project, Intelligent Fusion Technologies, Inc. and Rochester Institute of Technology propose a deep transfer learning approach to meet the USAF’s need. The proposed work is built upon the recent discriminative adversarial domain adaptation framework with the addition of one labeled image per target class. A key component of the proposed solution is a loss function that will guide the few labeled target features to move toward the targeted locations in the source domain. The proposed solution can be used in the following scenarios: i) transferring knowledge from simulated to measured data, ii) transferring from one modality (e.g., EO) to another (e.g., SAR), iii) transferring knowledge to new imaging conditions or sensors, and iv) transferring knowledge from one set of targets (e.g., sedan, SUV, pickup truck,…) to another set of targets (e.g., tank, LUV, military truck,…).