USA flag logo/image

An Official Website of the United States Government

Learning Estimates of Aggregate Performance (LEAP)

Award Information

Department of Defense
Air Force
Award ID:
Program Year/Program:
2009 / SBIR
Agency Tracking Number:
Solicitation Year:
Solicitation Topic Code:
Solicitation Number:
Small Business Information
Aptima, Inc.
12 Gill Street Suite 1400 Woburn, MA 01801-
View profile »
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Phase 1
Fiscal Year: 2009
Title: Learning Estimates of Aggregate Performance (LEAP)
Agency / Branch: DOD / USAF
Contract: FA8650-09-M-3928
Award Amount: $100,000.00


Intelligence, Surveillance and Reconnaissance (ISR) systems are becoming more and more complex, with ever increasing fidelity of data and ever increasing numbers of deployable UAVs and other sensors. Error rates are currently the best predictor of performance in autonomous, human, and sociotechnical systems. However, there are three key issues that arise when trying to determine the error rate of the complete ISR system: 1. learning algorithms must handle the sparse data available; 2. lack of structure in the data; and 3. complexity of overall human-automation system performance. Aptima proposes to address these challenges with a comprehensive human-system approach called Learning Estimates of Aggregate Performance (LEAP). The LEAP approach proposes to leverage techniques in both human modeling and machine learning to arrive at a solution that is both feasible and useful for calculating overall ISR system performance. LEAP leverages Signal Detection Theory to form an accurate model of human error and Relevance Vector Machines in order to reduce the necessary amount of training data. The LEAP approach, once achieved, has the ability to create more accurate performance estimates with much fewer human experiments necessary than with traditional learning approaches. BENEFIT: Drastic reduction the number of human experiments necessary in addition to leveraging available data to the fullest extent possible. Groundwork for human operator-based and human operator-model-based experimentation. Deep understanding of the important human factors that must be considered in the ISR system. Signal Detection Theory-informed models of human operator error and performance.

Principal Investigator:

Nathan Schurr
Human-Agent Collaboration Scientist

Business Contact:

Margaret J. Clancy
Chief Financial Officer
Small Business Information at Submission:

12 Gill Street Suite 1400 Woburn, MA 01801

EIN/Tax ID: 043281859
Number of Employees:
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No