You are here

Automated Image Analysis for Fisheries Applications


Video and Image recording systems are increasingly being used by NMFS for a multitude of applications.  Underwater systems are deployed on or near the seafloor or towed above the seafloor to record images of fish that are later analyzed to estimate numbers, sizes, and species composition of fish in an area.  Underwater systems are also installed in trawls to collect images that can be used to determine numbers, sizes, and species of fish caught in the trawl with the end goal of developing non-destructive trawls that collect all necessary information without actually catching the fish.  Other systems are installed on commercial fishing vessels to monitor what fish are caught, kept, and discarded during fishing operations.  The effort required to analyze data from these systems is time consuming and expensive.   Computer automated analysis to identify fish species contained in an image sequence or video segment has been moderately successful in very controlled photographic conditions when the potential number of species in the images is limited to just a few.  Fish lengths are successfully measured using stereo camera systems but require significant manual input by an analyst.  There is a need for innovative approaches to automated recognition and counting of fish species and estimation of length of fish in the images collected by these systems.


Project Goals:  The long term goal is to automate analysis of video and/or image sequences for two focal areas [(1) live fish underwater and (2) captured fish on vessels] to reduce the labor required to produce numerical data from the video or image sequences.  Each focal area presents a number of technical challenges.  Underwater images are frequently required to make use of ambient light to avoid influencing fish behavior and the resulting images are usually low contrast.  Fish in these images may be viewed from any aspect and distance from the camera.  Accurate counting of these fish requires tracking each individual during the time it is in the camera view to avoid counting one fish multiple times.  In contrast, captured fish could be imaged using artificial light at a fixed range but may (on a conveyor belt, for example) be positioned in any orientation in close proximity to, or partially obscured by, other fish.  Some fish species exhibit multiple color phases underwater and most fish change color and appearance with time after capture.  Successful projects will produce either software or hardware/software systems, applicable to one or more of these scenarios, that accept or collect sequences of images and count the number and sizes of each fish species present in the images.


Phase I Activities and Expected Deliverables:  This section should list the specific activities that will be carried out during the Phase I award.  It should also list the expected deliverables that must be met to address the project goals.  Phase I is the feasibility study.   Activities and deliverables should be written in bullet form.


For focal areas 1 (live fish underwater) and/or 2 (captured fish on vessels):


  • Identify potentially quantifiable features of commercially important and frequently encountered fish species occurring in the southeast US Atlantic Ocean, Caribbean Sea, and Gulf of Mexico that can be used for automated classification such as shape and color patterns.
  • Develop and demonstrate capability to automate data collection, potentially including but not necessarily limited to:

o   Identification of images or segments of video when fish are present

o   Species classification

o   Species-specific metrics of abundance and individual sizes

o   Habitat characteristics

  • Quantify error associated with data generated (e.g., proportion of fish correctly identified to species; degree of error about abundance or size estimates).  Demonstrate level of repeatability of results across multiple users
  • Deliverable: a detailed report documenting methods and results, with discussion of results and identification of successes and remaining challenges


Phase II Activities and Expected Deliverables:  (Same format as for Phase I.)  Phase II is the prototype development stage and where the majority of the R&D takes place.


For focal areas 1 (live fish underwater) and/or 2 (captured fish on vessels):


·         Develop one or more transferable software packages / platforms with user-friendly interface to accomplish data processing capabilities developed during Phase I activities

·         Products should allow improvement in species classification performance through incorporation of new training data and information on additional species.

·         Products should allow analyst intervention/correction in instances where confidence in species identification is low.

·         Desired analysis results include:

o   Individual fish length measurements and species identifications

o   Summary information on species composition and length distributions collected over multiple image sequences

o   Confidence intervals associated with individual species identifications and length measurements within a sequence and summary statistics for analysis of multiple sequences.

·         Deliverable: software package(s) / platform(s)

US Flag An Official Website of the United States Government