You are here

An Efficient I/O Framework for In Situ Data Extracts

Award Information
Agency: Department of Defense
Branch: Army
Contract: W912HZ-23-P-0022
Agency Tracking Number: A222-012-0032
Amount: $111,428.86
Phase: Phase I
Program: SBIR
Solicitation Topic Code: A22-012
Solicitation Number: 22.2
Timeline
Solicitation Year: 2022
Award Year: 2023
Award Start Date (Proposal Award Date): 2023-01-12
Award End Date (Contract End Date): 2023-10-11
Small Business Information
1712 Route 9 Suite 300
Clifton Park, NY 12065-3104
United States
DUNS: 010926207
HUBZone Owned: No
Woman Owned: No
Socially and Economically Disadvantaged: No
Principal Investigator
 Corey Wetterer-Nelson
 (306) 536-4181
 c.wetterer-nelson@kitware.com
Business Contact
 David Edsforth
Phone: (518) 881-4413
Email: david.edsforth@kitware.com
Research Institution
N/A
Abstract

Computational power has far outgrown modern I/O capabilities, leading to a bottleneck in high performance computing when attempting to write generated data to disk for future post-processing. In situ processing techniques mitigate this bottleneck by performing classically post-processing procedures while the simulation is running, without writing significant data to disk. Data extract artifacts can be generated in this way, storing reduced data to disk, still rich enough for further post-processing. However, data extracts are frequently generated with data on small subsets of processes, leading to imbalanced loading, straining I/O and reducing overall performance.  We propose the development of an automated data aggregation system targeting wildly load imbalanced in situ data extracts. This system will operate at scale, improving the I/O efficiency of extract-based in situ workflows. The design will be test driven from the beginning, focusing on quantitative performance metrics for future general and machine specific optimizations. Multiple in situ platforms will be targeted, including ParaView Catalyst and VisIt LibSim, ensuring wide availability of the data extract aggregation capabilities.

* Information listed above is at the time of submission. *

US Flag An Official Website of the United States Government