PARTEL: Periscope video Analysis using Reinforcement and TransfEr Learning

We propose a suite of video processing algorithms utilizing the machine learning (ML) techniques of artificial intelligence (AI) reinforcement learning, deep learning, and transfer learning to process submarine imagery obtained by means of periscope cameras. Machine learning (ML) can help in addressing the challenge of human failure of assessing the data of periscope imagery. Though pre-tuned black-box ML approaches can be used to train periscope imagery, the data available to train a black box is not robustly representative of the range of imagery expected to be encountered across all operating conditions. We address this problem using a combination of reinforcement learning and transfer learning where sufficient training data is not sufficient to support black box deep learning approaches. In prior work, Mayachitra has successfully applied reinforcement learning and reidentification approaches in maritime vessel detection from cameras in ships which will serve as representative imagery similar to periscope imagery data. Mayachitra has also worked on transfer learning where the last layers of pre-trained black box approaches are analyzed and then transferred to new data. In this STTR project, we will combine these approaches and use simultaneous cross platform reinforcement learning and transfer learning to address the problem of video processing on periscope imagery. Key metrics involve latency of vessel detection, time to identify, latency of vessel reacquisition after loss, rate of false positives, and rate of missed identifications. We will address the feasibility through modeling and analysis of the algorithms using representative imagery data, and include the initial design specifications and capabilities description to build a prototype solution in Phase II.