You are here

Index, Export and Search Archived Data for Enterprise Ground Satellite Command and Control Systems from Multiple Sources


TECHNOLOGY AREA(S): Space Platforms

OBJECTIVE: Develop techniques to index, export and search large volumes of archived data, across streams of telemetry and mission data and other data sources from multiple satellite missions in order to produce deep forensic analytics

DESCRIPTION: Technology breakthroughs have drastically increased the complexity of today’s satellites, with some satellites having more than of 10,000 satellite telemetry points for just a single satellite, updating at a cadence of once or more per second. In addition, communication technology has increased the data throughput capability across satellite links. The US alone has placed billions of dollars’ worth of assets into space and collecting, searching and extrapolating meaningful information from these assets quickly is a significantly important need. The net effect is that the amount of satellite data that must be downlinked to the ground has increased drastically. These amounts of data are overwhelming at the human level and searching them for patterns or actionable information becomes a challenge. The problem is compounded when applied across multiple satellite missions and beyond to the space enterprise, which results in streams of big amounts of data to search. Innovative software approaches which enable searching these large amounts of data in a fast and efficient way are therefore needed.

To be effective in a normal operational environment, the solution should be designed from the start with human centered computing in mind. The solution space should then have multiple automated processes that run in the background to ease big data workload. Big data storage and quick retrieval are also important. Indexing has proven to be a challenge for big data applications, but plays an integral role in ability to produce a timely and efficient search result. Solutions which reduce the time for index creation are desired. Detection and reporting processes both for real-time and after-the-fact analysis should be running in the background and not require substantial human interaction. It should be possible to conduct search queries in parallel and include ability to conduct multi-variable queries as combining results multiple mission areas. This will allow for powerful pattern matching and pattern discovery across missions. For example, such queries may be able to quickly identify problems in a particular ground area by searching multiple missions that fly over a particular location. The detection and reporting processes need to be self-sustaining, meaning that human management of these processes has been minimized. Satellite and payload state classification, indexing, and archival needs to be accomplished. Many processes should be running in the background including correlation between satellite and payload states with other data sources as well as attribution assessment. Humans should be able to monitor processes and set thresholds for human interaction in real time. Detection and reporting of events to humans with supporting correlations, likely attribution, and potential courses of action are the main real-time processes for human space system operators.

Innovative extensible and scalable low-cost software solutions are sought that will enable high performance searching and pattern and anomaly recognition. These software solutions should enable deep forensic analytics of large volumes of multiple satellite mission data from across the space enterprise. One approach could be a software application that indexes and searches large amounts of archived data from multiple satellite mission areas.

PHASE I: Conduct feasibility studies/technical analysis/simulation/proof-of-concept. The system should demonstrate the ability to work on a single satellite mission, but must scale support multiple missions. It is a requirement that if a software application approach is proposed, the software must be modular/opensource to allow for easy modifications in future increments. Demo prototype highly desirable.

PHASE II: Using the results from Phase I, construct, demonstrate and test tool with actual or properly simulated spacecraft data and other source data. Using simulated or actual data demonstrate a key finding through search of data across multiple missions. Recommend standards for representing satellite data for faster indexing.

PHASE III DUAL USE APPLICATIONS: Military Application: Transition to the RSC/MMSOC platform and then subsequently to the Enterprise Ground Service Framework.


  • Grolinger, Katarina, et al. "Challenges for mapreduce in big data." Services (SERVICES), 2014 IEEE World Congress on. IEEE, 2014.
  • Gandomi, Amir, and Murtaza Haider. "Beyond the hype: Big data concepts, methods, and analytics." International Journal of Information Management 35.2 (2015): 137-144.
  • Huijse, Pablo, et al. "Computational intelligence challenges and applications on large-scale astronomical time series databases." Computational Intelligence Magazine, IEEE 9.3 (2014): 27-39.
  • Roberts, Margaret E., Brandon M. Stewart, and Dustin Tingley. "Navigating the local modes of big data: The case of topic models." (2014).
  • Chen, Hsinchun, Roger HL Chiang, and Veda C. Storey. "Business Intelligence and Analytics: From Big Data to Big Impact." MIS quarterly 36.4 (2012): 1165-1188.
  • Marz, Nathan, and James Warren. Big Data: Principles and best practices of scalable realtime data systems. Manning Publications Co., 2015.
  • Faloutsos, Christos, and King-Ip Lin. FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. Vol. 24. No. 2. ACM, 1995.
  • “Movement Toward Common Satellite Ground System Gains, April 2015.

KEYWORDS: Big Data; indexing; searching big data; multiple mission satellite operations center; MMSOC; Satellite Command and Control (C2)


US Flag An Official Website of the United States Government