Description:
TECHNOLOGY AREA(S): Air Platform, Sensors, Electronics, Space Platforms
OBJECTIVE: To develop an automated software algorithm that can provide accurate temporal deformation characterization of a surface or target in the presence of three-dimensional relative camera motion
DESCRIPTION: DOD, DOE, and their contractors have an interest in dynamic digital photogrammetry as a remote sensing capability, particularly in environments where the ability to deploy sensitive ground sensors may be limited or there is a need to gather data over large areas. This can have applications for a variety of purposes such as change detection, battle damage assessment, explosive phenomenology, and post-test analysis of structural and weapon tests. The DTRA RD-TS Test Science and Technology Department has been working on the development of capabilities for performing dynamic digital photogrammetry using digital image correlation (DIC) from aerial platforms. Dynamic digital photogrammetry differs from the more common static photogrammetry (4,5,6) that uses only photos before and after an event to describe the total change. An example of an aerial platform would be two unmanned aerial systems (UAS) carrying high-speed cameras. DTRA RD-TS took advantage of recent developments in commercial off-the-shelf (COTS) DIC solutions to develop a capability that could provide an accurate temporal record of displacement at any point in the field-of-view during a dynamic event such as a buried explosion that creates ground deformation at the surface. The DIC estimates of ground deformation at the surface can then be compared to ground truth sensor information to evaluate their accuracy. However during the process of developing this capability, RD-TS experienced some challenges in small field of view characterization of small seismic sources (such as a sledgehammer or accelerated weight drop as viewed from ground-mounted cameras) and large field of view characterizations (20m x 20m or greater fields of view) of larger seismic sources from aerial platforms. One of the primary challenges was that many of the commercial software platforms for performing DIC, such as GOM Correlate Professional (2), which was used for initial RD-TS investigations, requires a pre-test calibration procedure to accurately characterize the lens parameters and camera positioning. This works well in a laboratory environment where there is no relative camera motion, but in field conditions there can be camera jitter from wind or small movements due to UAS position corrections that need to be accounted for (14,17,18). In this case, the DIC displacement errors increase as a function of time and also vary spatially over the field of view as the cameras move away from their initial calibrated position. Correction methods could include: multiple recalibrations over a shorter time period than the average timescale of significant relative motion when the calibration targets remain in the field of view; make adjustments to the images based on assumptions of non-moving points in the field of view prior to DIC analysis; or record the actual relative motion and make corrections to the DIC results or calibration files based on the known relative motion. With the technique RD-TS used to perform large field of view calibration with TRITOP and DIC analysis with GOM Correlate Professional (2), coded targets can be removed after calibration if no camera motion is expected. If only absolute motion is anticipated (e.g., cameras are connected by a camera bar) and there are motionless points in the field of view, results can be corrected using rigid body motion compensation. The purpose of this SBIR topic is to address the situation when there is relative camera motion, there may not be motionless points in the field of view, calibration targets or photomarkers may not be desirable in the field of view, and the surface of interest may be not be an ideal high-contrast speckled pattern (such as rock or geological surfaces)(11). In hostile territories, it also might not be possible to perform a pre-test calibration and a pre-calibrated camera/lens pair and motionless points in the field of view may be required (such as is required for the static photogrammetry software ShapeMetrix(5)). An automated software algorithm is required that can handle relative camera motion and can use known distances between easily identifiable features in the field of view for georeferencing.
PHASE I: During Phase I, an initial approach will be developed for designing an automated software algorithm that can be utilized with fields of view ranging from less than one square meter up to very large fields of view covering 10s to 100s of square meters. The algorithm needs to either be insensitive to relative motion or able to correct for it. This could involve adapting an existing commercial software (2,3), developing a new software algorithm (potentially starting from an open source DIC code (7)), or automating a static photogrammetry software (4,5,6) such as ShapeMetrix to work for dynamic photogrammetry. The algorithm needs to be bench tested with a simple dynamic scenario and two low-cost, high-resolution and reasonably high-speed cameras (minimum of 60 frames/sec although higher is preferred (19)). There needs to be a comparison of results when the cameras are motionless to results with relative camera motion. No aerial platform is necessarily required; the cameras can be moved by hand.
PHASE II: During Phase II, the digital image correlation software algorithm will be refined, documented, and will be tested in more realistic conditions. This will require one small-scale test using a small seismic source (such as a sledgehammer) with some method of obtaining ground truth. A second test needs to be performed using a larger field of view of at least 20m x 20m and a moving source with the same ground truth requirement. A fully successful Phase II would also allow for integration of cameras onto a wide range of aerial platforms ranging from UAS to satellites.
PHASE III: DUAL USE APPLICATIONS: In addition to defense applications, this method has broad academic applications (1,8,9,10) such as environmental change monitoring (10) , monitoring hazards such as landslides (16), or civil applications (9,15).
REFERENCES:
1: Colomina, I. and P. Molina (2014), Unmanned aerial systems for photogrammetry and remote sensing: A review, ISPRS Journal of Photogrammetry and Remote Sensing, 92, pp 79 – 97
2: Website: GOM Correlate, https://www.gom.com/3d-software/gom-correlate.html
3: Website: Xcitex – ProAnalyst Motion Analysis Software, https://www.xcitex.com/proanalyst-motion-analysis-software.php
4: Website: Pix4d: Professional photogrammetry and drone-mapping, https://www.pix4d.com/
5: Website: ShapeMetrix 3d, https://3gsn.at/produkte/shape-metrix/
6: Website: ADAM Technology, 3DM Analyst, https://www.adamtech.com.au/3dm/Analyst.html
7: Website: DICe Digital Image Correlation engine, Sandia National Laboratories, https://dice.sandia.gov (Open Source)
8: Vander Jagt, B., A. Lucieer, L. Wallace, D. Turner, and M. Durand (2015), Snow Depth Retrieval with UAS Using Photogrammetric Techniques, Geosciences, 5, 264-285
9: doi:10.3390d., /geosciences5030264
10: Reagan, D., A. Sabato, C. Niezrecki (2017), Unmanned aerial vehicle acquisition of three-dimensional digital image correlation measurements for structural health monitoring of bridges. In SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring, 1016909
11: Ferreira, E., J. Chandler, R. Wackrow, and K. Shiono (2017), Automated extraction of free surface topography using SfM-MVS photogrammetry, Flow Measurement and Instrumentation, 54, 243-249
12: Yu, J.H. and P.G. Dehmer (2012), Digital Image Correlation of Dynamic Impact Deformation Without Painted Dots Using ProAnalyst 3-D Photogrammetry Software, Army Research Laboratory ARL-TR-5913
13: Kedzierski, M. and P. Delis (2016), Fast Orientation of Video Images of Buildings Acquired from a UAV without Spabilization, Sensors, 16, 951
14: doi:10.3390/s16070951
15: Habib, A., I. Detchev, and E. Kwak (2014), Stability Analysis for a Multi-Camera Photogrammetric System, Sensors, 14, 15084-15112
16: doi:10.3390/s1408115084
17: Reich, M., J. Unger, F. Rottensteiner, and C. Heipke (2014), A new approach for an incremental orientation of micro-UAV image sequences
18: Unger, J., M. Reich, and C. Heipke (2014), UAV-based photogrammetry: monitoring of a building zone, The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5, ISPRS Technical Commission V Symposium, 23-25 June 2014, Riva del Garda Italy
19: Gomez, C. and H. Purdie (2016), UAV-based Photogrammetry and Geocomputing for Hazards and Disaster Risk Monitoring – A Review, Geoenvironmental Disasters, 3:23, doi: 10.1186/s40677-016-0060-y
20: Yang, Y., Z. Lin and F. Liu (2016), Stable Imaging and Accuracy Issues of Low-Altitude Unmanned Aerial Vehicle Photogrammetry Systems, Remote Sensing, 8, 316
21: doi: 10.3390/rs8040316
22: Miller, T.J., H.W. Schreier, and P. Reu (2007), High-speed DIC Analysis from a Shaking Camera System, in Society for Experimental Mechanics, 2007
23: Springfield, MA
24: Reu, P.L (2011), High/Ultra-high speed imaging as a diagnostic tool, Sandia National Laboratory, SAND2011-0978C
KEYWORDS: Remote Sensing, Photogrammetry, Digital Image Correlation
