USA flag logo/image

An Official Website of the United States Government

Multi-Modal Knowledge Acquisition from Documents

Award Information

Agency:
Department of Defense
Branch:
Navy
Award ID:
95103
Program Year/Program:
2010 / STTR
Agency Tracking Number:
N10A-019-0065
Solicitation Year:
N/A
Solicitation Topic Code:
NAVY 10T019
Solicitation Number:
N/A
Small Business Information
ObjectVideo
11600 Sunrise Valley Drive Suite # 210 Reston, VA 20191-
View profile »
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
 
Phase 1
Fiscal Year: 2010
Title: Multi-Modal Knowledge Acquisition from Documents
Agency / Branch: DOD / NAVY
Contract: N00014-10-M-0296
Award Amount: $69,908.00
 

Abstract:

Images with associated text are now available in vast quantities, and provide a rich resource for mining for the relationship between visual information and semantics encoded in language. In particular, the quantity of such data means that sophisticated machine learning approaches can be applied to determine effective models for objects, backgrounds, and scenes. Such understanding can then be used to: (1) understand, label, and index images that do not have text; and (2) augment the semantic understanding of images that do have text. This points to great potential power for searching, browsing, and mining documents containing image data. To this end, this STTR effort proposes a pipeline-based framework that focuses on the difficult task of text-image alignment (or correspondence). The proposed pipeline will take images and associated text to reduce correspondence ambiguity in stages. The framework will include both feed-forward and feed-back controls passing partially inferred information from one stage to another, leading to information enrichment and potential to provide inputs towards learning and understanding of novel objects and concepts. Ideas from both stochastic grammar representations and (joint) probabilistic representations will be investigated to facilitate modeling of text-image associations and visual modeling of objects, scenes, etc.

Principal Investigator:

Gaurav Aggarwal
Principal Investigator
7036549300
gaggarwal@objectvideo.com

Business Contact:

Paul Brewer
VP, NEW TECHNOLOGY
7036549314
pbrewer@objectvideo.com
Small Business Information at Submission:

ObjectVideo
11600 Sunrise Valley Drive Suite # 290 Reston, VA 20191

EIN/Tax ID: 541969286
DUNS: N/A
Number of Employees:
Woman-Owned: No
Minority-Owned: No
HUBZone-Owned: No
Research Institution Information:
University of Arizona
1040 E. 4th Street
Gould-Simpson Building
Tucson, AZ 85721
Contact: Kobus Barnard
Contact Phone: 5206214632