You are here

Optimization of Real Time Image Processing Techniques for Low Power Soldier and Unattended Ground Sensors


OBJECTIVE: To develop algorithmic approaches that address run-time code scheduling and internal processor chip resource allocation for active power management in Soldier borne and unattended ground systems real-time imaging. The algorithms must dynamically manage power consumption at all times given an image processing application, its processing load based on sensor frame rates of 30Hz or faster, input/output activity related to sending and or receiving various meta data, and current battery levels for mission applications where the Soldier carries the sensors. DESCRIPTION: Camera technology has improved to the point of being included in many defense applications including those carried by the Soldier and those left unattended at key observation points. Power, weight, and size are extremely important as these systems typically are powered by batteries. Thus, even a 2 Watt solution can cause Soldiers to be overburdened due to the number of batteries necessary for mission times. Even so, the imagery provided has been shown to be extremely valuable in the performance of the various missions. The quality and sheer volume of information from inclusion of cameras in many applications is such that no human can effectively consume it while working other primary tasks. Automation is being pushed to help alleviate information overload in order to effectively utilize the data provided, even though this would increase the power needed beyond that required for the camera technology. The automation covers many aspects from low level functions such as image non-uniformity correction to higher level functions such as object detection and tracking. The higher level functions are those that aid the operator in terms of workload. The processing algorithms used to create the higher level functions typically require many operations on each individual pixel before arriving at a result. The processing load, storage, and Input/Output (I/O) activities require processors with high sustained processing throughput coupled to large memories to keep up in real-time. Vehicle mounted applications are better able to manage the power necessary of processing systems required for the automation. Processing systems for Soldier mounted or unattended sensor applications have been more difficult to achieve due to power constraints imposed on them. Mission requirements for Soldier mounted or unattended sensor applications are scaled down from those vehicle mounted systems in terms of range capability, I/O, and functionality. Even so, more and more higher level functions are desired to be included in these systems. Power consumption and weight are critical. Real-time management of application code and hardware resources to optimize power is an unsolved problem. Technology in the commercial market tries to optimize power in terms of battery life over a given time period. For example smart phones optimize systems to perform a constant level of wireless I/O and display while trying to achieve arbitrarily time length before recharging is necessary. Typically the industry uses some common activity such as the length in hours of a flight between coasts. This is as opposed to actively minimizing power to the lowest level possible, but still maintaining processing throughput, to keep the activity going for as long as possible. Minimizing power can also lead to smaller, lighter sensor packages. The ability to adjust power in real-time, enabling close integration of modules with the camera is an unsolved problem. An innovative solution would in an integrated fashion dynamically adjust power as a function of processing load by analyzing the processing steps of the image processing algorithm and dynamically utilizing the hardware capability to selectively power down areas within a processor itself. Obviously this algorithmic approach taken for dynamic power adjustments must be implementable for demonstration and testing purposes. PHASE I: Develop an algorithmic approach to analyze image processing routines to minimize the power necessary for an overall application given a processor hardware architecture through a combination of code scheduling techniques and available processor hardware powering options. The goal is to minimize the power necessary to the fullest extent possible in order to reduce Soldier overburden. This initial algorithm should take advantage of existing hardware capability to power down selected sections of processor chips in real-time, including I/O ports to minimize required power. In order to test the algorithmic approach, develop a simple processor module test bed for initial algorithm testing. Integrate initial algorithms and test bed to demonstrate power reduction capability. Successful testing at the end of Phase 1 must show a level of algorithmic achievement such that potential Phase 2 development demands no major breakthroughs but would be a natural continuation and development of Phase 1 activity. The processor test bed for algorithmic demonstration could be based on FPGA technology if necessary. PHASE II: Complete algorithmic developments. Develop prototype processor test bed with on-board memories and I/O capable of acquiring digital data directly from a camera as well as capable of being tightly coupled to or inserted into a camera itself. System must demonstrate a 50% improvement in power using the algorithmic power approaches over the same hardware module without the algorithmic power approach. The Government will GFE a sample image processing algorithm for demonstration purposes. PHASE III: System will be utilizable in a soldier borne demonstration system at a U.S. Army test site. The algorithms and supporting test bed will also be utilizable in border protection by Department of Homeland Security. In private industry, the algorithms and processor test bed will be utilizable for law enforcement and all forms of perimeter security and protection. REFERENCES: 1. Chang, L.; Frank, D.J.; Montoye, R.K.; Koester, S.J.; Ji, B.L.; Coteus, P.W.; Dennard, R.H.; Haensch, W.;"Practical Strategies for Power-Efficient Computing Technologies,"Proceedings of the IEEE, vol. 98, no. 2, pp. 215-236, Feb. 2010 2. Gerndt, R.; Michalik, S.; Krupop, S.;"Embedded vision system for robotics and industrial automation,"Industrial Informatics (INDIN), 2011 9th IEEE International Conference on, pp. 895-899, 26-29 July 2011 3. Caulfield, J.T.; McCarley, P.L.; Elliott, J.; Massie, M.A.;"Bandwidth efficient sensor architectures with feature extraction,"Applied Imagery Pattern Recognition Workshop, 2008. AIPR'08. 37th IEEE, pp. 1-5, 15-17 Oct. 2008 4. Young-Geun Kim; Jayanthi, V.R.; In-So Kweon;"System-on-Chip Solution of Video Stabilization for CMOS Image Sensors in Hand-Held Devices,"Circuits and Systems for Video Technology, IEEE Transactions on, vol. 21, no. 10, pp. 1401-1414, Oct. 2011
US Flag An Official Website of the United States Government