Designing Large Data Handling Architectures

Award Information
Agency:
Department of Defense
Branch
n/a
Amount:
$1,246,335.00
Award Year:
2011
Program:
SBIR
Phase:
Phase II
Contract:
N00014-11-C-0523
Agency Tracking Number:
O2-1094
Solicitation Year:
2009
Solicitation Topic Code:
OSD09-SP4
Solicitation Number:
2009.2
Small Business Information
Harmonia, Inc.
2020 Kraft Drive, Suite 1000, Blacksburg, VA, -
Hubzone Owned:
Y
Minority Owned:
Y
Woman Owned:
Y
Duns:
556397615
Principal Investigator:
Marc Abrams
PI
(540) 951-5901
mabrams@harmonia.com
Business Contact:
Pallabi Saboo
CEO
(540) 951-5915
psaboo@harmonia.com
Research Institution:
Stub




Abstract
Harmonia proposes to continue development from Phase I of a Large Data Handling Architecture (LDHA). Given sufficient amounts of commodity hardware, its goal is to scale up to ingest a terabyte or more of data per hour from each open source or sensor; store tens of thousands and more of terabyte files; and support operations on databases that use complex structures as table cells, sparse tables, and billions or more rows and a million or more columns. Our implementation uses the Ubuntu server Linux distribution, Hadoop core, HBase database and Chukwa data collector. We implement ingesters as Java Servlets to collect continuously and automatically pass for insertion into Hadoop a variety of open sources including, but not limited to structured and unstructured text, sensor data, images, and streaming video, experimenting with real continuous feeds available on the Internet. We develop end user programming tools that empower analysts to define analysis tasks for distributed execution even though they do not have a programming background in MapReduce. Our architecture is service oriented, high performance, extensible, and permits integration with the DoD Global Information Grid. We support disadvantaged users through services that allow downloading of data. We do extensive performance assessment of bottlenecks.

* information listed above is at the time of submission.

Agency Micro-sites


SBA logo

Department of Agriculture logo

Department of Commerce logo

Department of Defense logo

Department of Education logo

Department of Energy logo

Department of Health and Human Services logo

Department of Homeland Security logo

Department of Transportation logo

Enviromental Protection Agency logo

National Aeronautics and Space Administration logo

National Science Foundation logo
US Flag An Official Website of the United States Government