Designing Large Data Handling Architectures

Award Information
Agency: Department of Defense
Branch: Navy
Contract: N00014-11-C-0523
Agency Tracking Number: O2-1094
Amount: $1,246,335.00
Phase: Phase II
Program: SBIR
Awards Year: 2011
Solicitation Year: 2009
Solicitation Topic Code: OSD09-SP4
Solicitation Number: 2009.2
Small Business Information
2020 Kraft Drive, Suite 1000, Blacksburg, VA, -
DUNS: 556397615
HUBZone Owned: Y
Woman Owned: Y
Socially and Economically Disadvantaged: Y
Principal Investigator
 Marc Abrams
 (540) 951-5901
Business Contact
 Pallabi Saboo
Title: CEO
Phone: (540) 951-5915
Research Institution
Harmonia proposes to continue development from Phase I of a Large Data Handling Architecture (LDHA). Given sufficient amounts of commodity hardware, its goal is to scale up to ingest a terabyte or more of data per hour from each open source or sensor; store tens of thousands and more of terabyte files; and support operations on databases that use complex structures as table cells, sparse tables, and billions or more rows and a million or more columns. Our implementation uses the Ubuntu server Linux distribution, Hadoop core, HBase database and Chukwa data collector. We implement ingesters as Java Servlets to collect continuously and automatically pass for insertion into Hadoop a variety of open sources including, but not limited to structured and unstructured text, sensor data, images, and streaming video, experimenting with real continuous feeds available on the Internet. We develop end user programming tools that empower analysts to define analysis tasks for distributed execution even though they do not have a programming background in MapReduce. Our architecture is service oriented, high performance, extensible, and permits integration with the DoD Global Information Grid. We support disadvantaged users through services that allow downloading of data. We do extensive performance assessment of bottlenecks.

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government