AS THE DOD IS MOVING TOWARDS THE LARGE-SCALE USE OF ARTIFICIAL INTELLIGENCE SYSTEMS FOR SUPPORTING DECISION-MAKING AND INFORMATION ANALYSIS, IT BECOMES INCREASINGLY IMPORTANT THAT WE DEVELOP MECHANISMS FOR ASSESSING THE PERFORMANCE OF THESE SYSTEMS.

Award Information
Agency:
Department of Defense
Branch
Defense Advanced Research Projects Agency
Amount:
$49,992.00
Award Year:
1986
Program:
SBIR
Phase:
Phase I
Contract:
n/a
Award Id:
3597
Agency Tracking Number:
3597
Solicitation Year:
n/a
Solicitation Topic Code:
n/a
Solicitation Number:
n/a
Small Business Information
201 San Antonio Circle - Ste 2, Mountain View, CA, 94040
Hubzone Owned:
N
Minority Owned:
N
Woman Owned:
N
Duns:
n/a
Principal Investigator:
DR R M TONG
(415) 941-3912
Business Contact:
() -
Research Institution:
n/a
Abstract
AS THE DOD IS MOVING TOWARDS THE LARGE-SCALE USE OF ARTIFICIAL INTELLIGENCE SYSTEMS FOR SUPPORTING DECISION-MAKING AND INFORMATION ANALYSIS, IT BECOMES INCREASINGLY IMPORTANT THAT WE DEVELOP MECHANISMS FOR ASSESSING THE PERFORMANCE OF THESE SYSTEMS. WE BELIEVE THAT ASKING "HOW INTELLIGENT IS THE SYSTEM?" IS NOT THE CORRECT PERFORMANCE QUESTION AND THAT MORE APPROPRIATE ARE "DOES THE SYSTEM HELP THE USER?" "IN WHAT WAY?" AND "HOW CAN THIS BE MEASURED?" OUR APPROACH IS BASED ON THE PREMISE THAT THE USERS, DEVELOPERS, MANAGERS AND FUNDERS OF THESE SYSTEMS HAVE "EXPERT" KNOWLEDGE ABOUT PERFORMANCE AND EVALUATION THAT NEEDS TO BE CAPTURED IF A COMPREHENSIVE EVALUATION IS TO BE PERFORMED. THE METHODOLOGY WE PROPOSE STRUCTURES THIS KNOWLEDGE BY USING A HIERARCHY OF EVALUATION CONTEXTS, USES A RULE-BASED LANGUAGE FOR EXPRESSING THE KNOWLEDGE, AND EMPLOYS THE MATHEMATICS OF FUZZY SET THEORY TO MODEL THE INHERENT VAGUENESS IN THE KNOWLEDGE. THE PROPOSAL CONTAINS AN OVERVIEW OF OUR EVALUATION METHODOLOGY AND ILLUSTRATES IT WITH A SIMPLE EXAMPLE. WE ALSO DISCUSS THE NEED TO DESIGN A SET OF TOOLS TO HELP OPERATIONALIZE THE METHODOLOGY, AND CONSIDER SOME PILOT EXPERIMENTS THAT SHOULD BE PERFORMED TO VALIDATE THE IDEAS.

* information listed above is at the time of submission.

Agency Micro-sites


SBA logo

Department of Agriculture logo

Department of Commerce logo

Department of Defense logo

Department of Education logo

Department of Energy logo

Department of Health and Human Services logo

Department of Homeland Security logo

Department of Transportation logo

Enviromental Protection Agency logo

National Aeronautics and Space Administration logo

National Science Foundation logo
US Flag An Official Website of the United States Government