Kiwifruit detection in field images using Faster R-CNN with ZFNet

摘要

A kiwifruit detection system for field images was developed based on the deep convolutional neural network, which has a good robustness against the subjectivity and limitation of the features selected artificially. Under different lighting conditions, 2,100 sub-images with 784×784 pixels were prepared by random sub-sampling from 700 field captured images with a pixel resolution of 2352×1568 pixels. Sub-images were used as network training and validation samples. A faster R-CNN was trained end-to-end by using back-propagation and stochastic gradient descent techniques with Zeiler and Fergus network (ZFNet). The average precision of the Faster R-CNN-based kiwifruit detector was 89.3%. Finally, another 100 images of kiwifruit canopies in the field environment (including 5,918 fruits) were used for testing the network. The test results showed that the recognition ratio of occluded fruit, overlapping fruit, adjacent fruit and separated fruit were 82.5%, 85.6%, 94.3% and 96.7%, respectively. Overall, the model reached a recognition rate of 92.3%. The technique took 0.274 s to process each image (for images with 2352×1568 pixels) and only 5.0 ms on average to detect a fruit. Comparing against the conventional methods, it suggested that the proposed method has higher recognition rate and faster speed. Especially, the proposed technique was able to simultaneously detect individual kiwifruit in clusters, which provides a promise for accurate yield mapping and multi-arm robotic harvesting.