Apple fruit recognition algorithm based on multi-spectral dynamic image analysis

Juan Feng, Lihua Zeng, Long He

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The ability to accurately recognize fruit on trees is a critical step in robotic harvesting. Many researchers have investigated a variety of image analysis methods based on different imaging technologies for fruit recognition. However, challenges still occur in the implementation of this goal due to various factors, especially variable light and proximal color background. In this study, images with fruit were acquired with a Forward Looking Infrared (FLIR) camera based on the Multi-Spectral Dynamic Imaging (MSX) technology. In view of its imaging mechanism, the optimal timing and shooting angle for image acquisition were pre-analyzed to obtain the maximum contrast between fruit and background. An effective algorithm was developed for locking potential fruit regions, which was based on the pseudo-color and texture information from MSX images. The algorithm was applied to 506 training and 340 evaluating images, including a variety of fruit and complex backgrounds. Recognition precision and sensitivity of these complete fruit regions were both above 92%, and those of incomplete fruit regions were not lower than 72%. The average processing time for each image was less than 1 s. The results indicated that the developed algorithm based on MSX imaging was effective for fruit recognition and could be suggested as a potential method for the automation of orchard production.

Original languageEnglish (US)
Article number949
JournalSensors (Switzerland)
Volume19
Issue number4
DOIs
StatePublished - Feb 2 2019

Fingerprint

fruits
Malus
Fruits
image analysis
Image analysis
Fruit
Imaging techniques
Color
orchards
Technology
Orchards
color
Image acquisition
Automation
Robotics
automation
robotics
locking
acquisition
education

All Science Journal Classification (ASJC) codes

  • Analytical Chemistry
  • Atomic and Molecular Physics, and Optics
  • Biochemistry
  • Instrumentation
  • Electrical and Electronic Engineering

Cite this

@article{410f18c5eaf0455495672142e3a6e16c,
title = "Apple fruit recognition algorithm based on multi-spectral dynamic image analysis",
abstract = "The ability to accurately recognize fruit on trees is a critical step in robotic harvesting. Many researchers have investigated a variety of image analysis methods based on different imaging technologies for fruit recognition. However, challenges still occur in the implementation of this goal due to various factors, especially variable light and proximal color background. In this study, images with fruit were acquired with a Forward Looking Infrared (FLIR) camera based on the Multi-Spectral Dynamic Imaging (MSX) technology. In view of its imaging mechanism, the optimal timing and shooting angle for image acquisition were pre-analyzed to obtain the maximum contrast between fruit and background. An effective algorithm was developed for locking potential fruit regions, which was based on the pseudo-color and texture information from MSX images. The algorithm was applied to 506 training and 340 evaluating images, including a variety of fruit and complex backgrounds. Recognition precision and sensitivity of these complete fruit regions were both above 92{\%}, and those of incomplete fruit regions were not lower than 72{\%}. The average processing time for each image was less than 1 s. The results indicated that the developed algorithm based on MSX imaging was effective for fruit recognition and could be suggested as a potential method for the automation of orchard production.",
author = "Juan Feng and Lihua Zeng and Long He",
year = "2019",
month = "2",
day = "2",
doi = "10.3390/s19040949",
language = "English (US)",
volume = "19",
journal = "Sensors",
issn = "1424-3210",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "4",

}

Apple fruit recognition algorithm based on multi-spectral dynamic image analysis. / Feng, Juan; Zeng, Lihua; He, Long.

In: Sensors (Switzerland), Vol. 19, No. 4, 949, 02.02.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Apple fruit recognition algorithm based on multi-spectral dynamic image analysis

AU - Feng, Juan

AU - Zeng, Lihua

AU - He, Long

PY - 2019/2/2

Y1 - 2019/2/2

N2 - The ability to accurately recognize fruit on trees is a critical step in robotic harvesting. Many researchers have investigated a variety of image analysis methods based on different imaging technologies for fruit recognition. However, challenges still occur in the implementation of this goal due to various factors, especially variable light and proximal color background. In this study, images with fruit were acquired with a Forward Looking Infrared (FLIR) camera based on the Multi-Spectral Dynamic Imaging (MSX) technology. In view of its imaging mechanism, the optimal timing and shooting angle for image acquisition were pre-analyzed to obtain the maximum contrast between fruit and background. An effective algorithm was developed for locking potential fruit regions, which was based on the pseudo-color and texture information from MSX images. The algorithm was applied to 506 training and 340 evaluating images, including a variety of fruit and complex backgrounds. Recognition precision and sensitivity of these complete fruit regions were both above 92%, and those of incomplete fruit regions were not lower than 72%. The average processing time for each image was less than 1 s. The results indicated that the developed algorithm based on MSX imaging was effective for fruit recognition and could be suggested as a potential method for the automation of orchard production.

AB - The ability to accurately recognize fruit on trees is a critical step in robotic harvesting. Many researchers have investigated a variety of image analysis methods based on different imaging technologies for fruit recognition. However, challenges still occur in the implementation of this goal due to various factors, especially variable light and proximal color background. In this study, images with fruit were acquired with a Forward Looking Infrared (FLIR) camera based on the Multi-Spectral Dynamic Imaging (MSX) technology. In view of its imaging mechanism, the optimal timing and shooting angle for image acquisition were pre-analyzed to obtain the maximum contrast between fruit and background. An effective algorithm was developed for locking potential fruit regions, which was based on the pseudo-color and texture information from MSX images. The algorithm was applied to 506 training and 340 evaluating images, including a variety of fruit and complex backgrounds. Recognition precision and sensitivity of these complete fruit regions were both above 92%, and those of incomplete fruit regions were not lower than 72%. The average processing time for each image was less than 1 s. The results indicated that the developed algorithm based on MSX imaging was effective for fruit recognition and could be suggested as a potential method for the automation of orchard production.

UR - http://www.scopus.com/inward/record.url?scp=85064936083&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064936083&partnerID=8YFLogxK

U2 - 10.3390/s19040949

DO - 10.3390/s19040949

M3 - Article

VL - 19

JO - Sensors

JF - Sensors

SN - 1424-3210

IS - 4

M1 - 949

ER -