Algorithm Performance Evaluation with Low Sample Size

The team of Signature Research, Inc. and Michigan Technological University will develop and demonstrate methods and metrics to evaluate the performance of machine learning-based computer vision algorithms with low numbers of samples of labeled EO imagery. We will use the existing xView panchromatic dataset to demonstrate a proof-of-concept set of tools. If successful, in Phase II, we will extend the tools and techniques to other signature spectra. The demonstrated capability will include variation across at least two operating conditions, e.g., geographic diversity and object size. We will use two methodologies to try to solve this difficult and complex problem. The two methods will include Network Stability Theory and Mutual Information Theory. In addition to the small labelled dataset, we will augment the testing of the tools with variability in the signatures using SGR-generated synthetic imagery. The Phase I program will result in proof-of-concept performance assessment on the selected data set.