You are here

Brain-Inspired Few-Shot Object Recognition


TECHNOLOGY AREA(S): Info Systems, 

OBJECTIVE: Develop a brain-inspired artificial neural network algorithm that performs accurate and robust object detection. 

DESCRIPTION: Artificial Intelligence has yet to surpass the human brain in terms of training time: even the best algorithms require huge datasets that train carefully-tuned models over a long period of time. Current state-of-the-art artificial neural networks for image identification, called Convolutional Neural Networks (CNNs), are achieving at, or higher, than human level performance in recognizing 2D images from the open source ImageNet database ( of labeled images (plant, animals, etc.) which benchmarks performance. CNN performance in ImageNet data for standard color photos, greyscale, and photos based on textures (i.e.; elephant skin) are on par or slightly better than human performance. CNN performance degrades substantially with images of object silhouettes (black object with white background) and edges (image features represented with only lines), when objects under observation are small in scale relative to surrounding area, and when object viewpoint, rotation, size, and illumination vary. CNN training on ImageNet data requires on the order of 1000 examples per object class yet humans need to see a new object only once or twice and it becomes instantly recognized at a later time. We are seeking brain-inspired artificial neural network algorithms that can meet the performance objectives of recognizing objects in images from less than 10 training examples with 90% confidence of object identification under a full range of image observation conditions to include varying scale, size, illumination (full sunlight to low light), occlusion (from zero to 90% in both height/width increments of 15%), and rotation (in increments of 30°). A virtual 3D environment to train and demonstrate viability of the proposed algorithm is desired such as the Unity open source game engine ( It is desired that artificial neural network algorithm be developed with open source development code such as TensorFlow ( or Python ( It is desired to have a high resolution color video camera ( with a minimum of 3840 x 2160 pixels be used to observe raw pixels from the 3D virtual environment to train and demonstrate feasibility and performance of the algorithm. Novel approaches to train for object recognition that realistically emulate the human vision system (e.g., stereopsis, foveation, etc.) are desired if a breakthrough in capability is feasible. 

PHASE I: Demonstrate in the Unity game engine environment an innovative and beyond state-of-the-art approach that demonstrates a viable and feasible technical approach to meet the topic objectives. 

PHASE II: Develop, demonstrate, and test object recognition algorithms that meet the topic objective. Object recognition algorithms will be tested incrementally against the Common Objects in Context (COCO),, and ImageNet,, image data sets to establish benchmark performance against state-of-the-art. Image data sets will be developed to be capable of being rendered to meet the topic objectives for challenging object recognition training observations (scale, illumination, etc.). Object recognition algorithms will be matured to a TRL 5 by the end of Phase II. 

PHASE III: Robust object detection would be a boon for safer commercial autonomous self-driving vehicles, drones, and robots. Military applications would include target identification, combat friend or foe identification, and for live force-on-target training at combat training centers 


1: D. Tsao, "How the Brain’s Face Code Might Unlock the Mysteries of Perception", , December 11, 2018

KEYWORDS: Object Recognition, Convolutional Neural Network, And Human Vision 

US Flag An Official Website of the United States Government