TeleSculptor Multiplex: Fusing Multiple FMV Feeds in 3D

Multiple concurrent FMV feeds can provide useful context during special forces missions but the flood of data from disconnected viewpoints can dramatically increase cognitive load on SOF operators. Substantial network bandwidth is also consumed by streaming multiple concurrent video feeds with highly redundant content. Kitware proposes TeleSculptor Multiplex, an approach to combine multiple FMV feeds into a coherent 3D video for web-based display using OGC standards. Our approach builds on our prior work in 3D shape estimation from FMV. We will recover 3D shape from the multiple FMV feeds at multiple scales, align this data, and build a unified scene model. We will extract static texture and dynamic texture (e.g. movers) into separate layers that we will update at different rates for more efficient storage and transmission. We will stream this 3D video display to 3D geospatial contexts on mobile and web clients (e.g. ATAK and Cesium) using existing standards such as CDB and 3D Tiles. All our work is provided with unlimited rights to the government and will be made open source, if allowed, to maximize adoption of the technology.