USA flag logo/image

An Official Website of the United States Government

Automated Diagnosis of Usability Problems Using Statistical Computational…

Award Information

Department of Defense
Air Force
Award ID:
Program Year/Program:
2003 / STTR
Agency Tracking Number:
Solicitation Year:
Solicitation Topic Code:
Solicitation Number:
Small Business Information
Aptima, Inc.
12 Gill Street Suite 1400 Woburn, MA -
View profile »
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Phase 1
Fiscal Year: 2003
Title: Automated Diagnosis of Usability Problems Using Statistical Computational Methods
Agency / Branch: DOD / USAF
Contract: F49620-03-C-0067
Award Amount: $100,000.00


The effects of poor usability range from mere inconvenience to disaster. Human factors specialists employ usability analysis to reduce the likelihood or impact of such failures. However, good usability analysis requires usability reports that are rarelycollected, rarely complete, and difficult to analyze.Aptima and the Center for Intelligent Information Retrieval (U. Mass. Amherst) have partnered to develop a usability analysis system that addresses these problems. The system will consist of (1) an interface to elicit useful usability reports in naturallanguage, (2) a text analysis engine that classifies these reports (or existing usability reports) using a validated taxonomy, and (3) an analysis interface for analyzing individual usability reports and trends in usability problems. We will train and testthe system using a very large corpus of publicly available usability reports categorized into an extension of the User Action Framework. Aptima and CIIR will deliver a report of results, a software prototype, and a corpus of manually categorized usabilityreports. The anticipated benefits of the proposed Usability Reports Diagnostic Tool are as follows. The proposed tool will help to collect and classify usability reports. During usability tests, the tool will facilitate the articulation of observedusability problems. The tool's automatic classifier will analyze the resulting usability reports using a text analysis algorithm and categorize them according to a classification schema (also to be developed under this project). The output will identifythe problem observed, classify it by type and by cause, and allow further refine by the user.The Usability Reports Diagnostic Tool will be useful for both usability professionals within software development organizations who must generate reports for internal use and customers, and human factors or other usability professionals in customerorganizations who are evaluating the usability of the products. Our solution will enable users (and observers such as usability engineers) to express usability problems in their own terms, while systematically categorizing those problems in meaningful wayfor designers.

Principal Investigator:

Jared Freeman
Principal Cognitive Scientist

Business Contact:

Margaret J. Clancy
Chief Financial Officer
Small Business Information at Submission:

Aptima, Inc.
12 Gill Street, Suite 1400 Woburn, MA 01801

EIN/Tax ID: 043281859
Number of Employees:
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Research Institution Information:
408 Goodell, 140 Hicks Way
Amherst, MA 01003
Contact: Carol P. Sprague
Contact Phone: (413) 545-0698
RI Type: Nonprofit college or university