Target Tracking via Deep Learning

Persistent tracking of high-value targets is of great interest for reconnaissance and surveillance applications. In recent work, deep neural networks have demonstrated excellent performance on the popular Visual Object Tracking (VOT) challenge; however, these algorithms have not been tested on applications of interest to the Air Force, such as ground vehicle tracking in video recorded from Unmanned Aerial Vehicles (UAVs). It is still unclear which algorithms will be effective in such use-cases. Moreover, the efficacy of deep learning for video tracking is still ambiguous. The Toyon team proposes an investigation of recent Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) based algorithms, for video tracking, with a goal of eventual use for real-time Wide-Area Surveillance applications. These include hybrid approaches (CNNs+RNNs) such as an extension to the GOTURN and ROLO algorithms. Additionally, a novel approach to tracking will be developed, using visual features extracted from CNNs and the prediction capability of RNNs in a Bayesian framework, for track forecasting. These approaches will be benchmarked against other state-of-the-art algorithms and Toyons VideoPlus (Aware) tracking software, to investigate the utility of deep learning in video tracking, and explore potential avenues of improvement.