Automated Diagnosis of Usability Problems Using Statistical Computational Methods

Award Information
Agency: Department of Defense
Branch: Air Force
Contract: F49620-03-C-0067
Agency Tracking Number: F033-0311
Amount: $100,000.00
Phase: Phase I
Program: STTR
Awards Year: 2003
Solicitation Year: N/A
Solicitation Topic Code: N/A
Solicitation Number: N/A
Small Business Information
12 Gill Street, Suite 1400, Woburn, MA, 01801
DUNS: 967259946
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: N
Principal Investigator
 Jared Freeman
 Principal Cognitive Scientist
 (202) 842-1548
 freeman@aptima.com
Business Contact
 Margaret Clancy
Title: Chief Financial Officer
Phone: (781) 496-2415
Email: clancy@aptima.com
Research Institution
 UNIV. OF MASSACHUSETTS
 Carol P Sprague
 408 Goodell, 140 Hicks Way
Amherst, MA, 01003
 (413) 545-0698
 Nonprofit college or university
Abstract
The effects of poor usability range from mere inconvenience to disaster. Human factors specialists employ usability analysis to reduce the likelihood or impact of such failures. However, good usability analysis requires usability reports that are rarelycollected, rarely complete, and difficult to analyze.Aptima and the Center for Intelligent Information Retrieval (U. Mass. Amherst) have partnered to develop a usability analysis system that addresses these problems. The system will consist of (1) an interface to elicit useful usability reports in naturallanguage, (2) a text analysis engine that classifies these reports (or existing usability reports) using a validated taxonomy, and (3) an analysis interface for analyzing individual usability reports and trends in usability problems. We will train and testthe system using a very large corpus of publicly available usability reports categorized into an extension of the User Action Framework. Aptima and CIIR will deliver a report of results, a software prototype, and a corpus of manually categorized usabilityreports. The anticipated benefits of the proposed Usability Reports Diagnostic Tool are as follows. The proposed tool will help to collect and classify usability reports. During usability tests, the tool will facilitate the articulation of observedusability problems. The tool's automatic classifier will analyze the resulting usability reports using a text analysis algorithm and categorize them according to a classification schema (also to be developed under this project). The output will identifythe problem observed, classify it by type and by cause, and allow further refine by the user.The Usability Reports Diagnostic Tool will be useful for both usability professionals within software development organizations who must generate reports for internal use and customers, and human factors or other usability professionals in customerorganizations who are evaluating the usability of the products. Our solution will enable users (and observers such as usability engineers) to express usability problems in their own terms, while systematically categorizing those problems in meaningful wayfor designers.

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government