You are here

Speedy Sparse Bundle Adjustment for Video/Image Sequences


OBJECTIVE: Develop robust, rapid, and automated Sparse Bundle Adjustment software to create and update the camera extrinsic and intrinsic parameters for a set of input images or video frames faster than the current state of the art. DESCRIPTION: Sparse bundle adjustment (SBA) has been a solved problem since the 1960's. Modern computers have made the SBA process computationally tractable and it is particularly useful for 3-D reconstruction to determine the extrinsic and intrinsic parameters of a moving camera (1). Automatic determination of extrinsic and intrinsic parameters of a camera will result in more autonomy of surveillance systems and deliver better data to the war fighter. The state of the art implementations are Bundler (4) and Visual Structure from Motion (2) which utilizes multi-core bundle adjustment (MCBA). These implementations work well but they are slow because they do not assume any spatial or temporal order to the input images. For Air Force applications the images are likely to be in a logical spatial-temporal order therefore we hope we can gain a tremendous speed improvement by processing images in order. A robust fully automatic speedy sparse bundle adjustment algorithm will help the USAF operate surveillance systems in GPS denied airspace plus automate the process of building a camera model. Modeling a camera is a time consuming process that often requires a specialized skill set hence an automatic camera model process is desired. We are looking for an open technology development (5) that builds on the existing sparse bundle adjustment (SBA) foundational work which makes dramatic performance improvements. The offeror should consider implementations that will utilize graphics processing unit (GPU) acceleration (either OpenCL or CUDA) so the software will run on commodity CPU(s) or GPU(s). A flexible/robust system is the desired end result, thus successful offerors will allow the software to use a multitude of existing feature detection algorithms such as scale-invariant feature transform (SIFT), speeded up robust features (SURF), and oriented fast rotated brief (ORB), etc. Output is a simple camera matrix with typical values for the camera position, orientation (extrinsics) and focal length, principal point, and distortion factor (intrinsics). The proposal should specify datasets used for both full motion video (FMV) and wide area motion imagery (WAMI) sensors. The government will provide potential baseline datasets on the public release site of the AFRL Sensor Data Management System (3). PHASE I: The expected product of Phase I is a speedy sparse bundle adjustment algorithm that takes as input a single stream of EO visible airborne imagery and builds the camera matrix for each frame (both intrinsic and extrinsic) documented in a final report and implemented in a proof-of-concept software deliverable. PHASE II: The expected product of Phase II is a prototype implementation of the Phase I proof of concept algorithm, enhanced to include faster run times by operating on both multiple CPUs and multiple GPUs. Report/illustrate demonstrated improvements. PHASE III: DUAL USE COMMERCIALIZATION: Military Application: Visualization, training, combat search and rescue. Commercial Application: Mapping and navigation, city planning, emergency response. REFERENCES: 1. Jeong, Y., Nister, D., Steedly, D., Szeliski, R., and Kweon, I.,"Pushing the Envelope of Modern Methods for Bundle Adjustment", 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence. 2. Wu, ChangChang,"VSFM: Visual Structure from Motion", 2012, 3. AFRL/RYA"SDMS: Sensor Data Management System", 2012, 4. Snavely, N,"Bundler: Structure from Motion (SfM) for Unordered Image Collections", 2012 5. DoD CIO"s Office,"Open Technology Development (OTD): Lessons Learned & Best Practices for Military Software", 2011,
US Flag An Official Website of the United States Government