Fruit visibility analysis for robotic citrus harvesting.

摘要

Automating orange harvesting in Florida could support the citrus industry in the face of a decreasing labour force and global market competition. Fruit recognition is the 1st critical operation in robotic harvesting, and fruit visibility in the tree canopy poses a challenge to fruit detection. Fruit trees such as oranges have a dense canopy, which can often result in partial or complete occlusion of fruits. Fruit visibility and approaches to increase visibility of oranges by viewing the canopy with different camera perspectives was investigated. Fruit visibility was defined as the ratio of the number of fruits visible to a human observer to the total number of fruits inside a region of interest (ROI), which was a vol. of tree canopy enclosed by a 0.125 m3 bounding cube. Multiple images of ROI were acquired using orthographic viewing and multiple-perspective viewing. Orthographic viewing involved taking the 6 orthographic views perpendicular to the ROI faces, while multiple-perspective viewing acquired 9 different perspectives at combinations of 45 degrees angles to the ROI front face. Sets of orthographic and multiple-perspective images were obtained from a commercial orange grove in Florida. Combining visible fruits from multiple-perspective images yielded a fruit visibility of 0.91 compared to 0.82 from combined orthographic images. In addition, an image processing fruit recognition algorithm detected 0.87 of visible fruits in the ROI using the multiple-perspective images. These fruit visibility levels show a substantial improvement compared to results from previous literature, which reported 0.40-0.70 fruit visibility for citrus trees. Integrating the multiple-perspective viewing approach into a fruit exploration function of a harvesting robot could improve overall harvesting efficiency.