High-Throughput Image Analysis Framework for Fruit Detection, Localization and Measurement from Video Streams

摘要

Food crises and security issues are getting worse due to climate change and population growth. One of the solutions being sought is the use of efficient breeding systems, which requires accurate and detailed phenotyping offruits and plants. However, traditional phenotyping methods are time consuming, labor intensive and prone to human error. Therefore, measuring the morphological and physiological parameters of fruits automatically is highly recommended. In this work, a high-throughput technique forfruits detection, localization and measurement from video streams using computer vision and deep neural networks is proposed. In contrast with other works that were developed for single type offruits, a versatile method is proposed herein that can be applied for different types offruits using a vision system to scan through plants row by row in a greenhouse. A real-time object detection algorithm using YOLOv2, a deep neural network-based detector, is used for fruit detection and localization on video frames with a hit rate of 84.98%. An individual fruit tracking algorithm is applied throughout the video stream to perform tracking of multiple fruits. The online tracking algorithm includes feature matching, optical flow and projective transformation optimized by occlusion handling techniques such as by applying threshold indices and denoising. On the other hand, the offline tracking algorithm uses a voting method to reduce the false alarms caused by the object detector. Finally, phenotyping informationsuch as fruit counts, ripening stage, fruit size, and 2D spatial distribution maps were obtained. The proposed framework has demonstrated its efficacy in obtaining satisfactory phenotyping information that is usefulfor production management as well as its potential utilization in robotic operations.