Toward Semantic Action Recognition for Avocado Harvesting Process based on Single Shot MultiBox Detector

摘要

To date, human action recognition is still a challenging topic and has been addressed from many perspectives. Detection of human actions can be useful to obtain relevant information to improve complex processes, which is the case of agricultural applications. In this work, the detection of objects that can provide information for human action recognition based on semantic representations is studied. For this purpose, a convolutional neuronal network based on Single Shot MultiBox Detector meta-architecture and MobileNet feature extractor was implemented, which has been trained to detect nine classes of objects during the process of collecting avocados in a Chilean farm. We have found that such detected objects are related to seven possible actions that can be detected during avocado harvesting process. Such information could allow to directly detect certain actions in still images, or improve conventional action detection methods during the harvesting process. The results show that is possible to detect human actions during the process, obtaining action recognition performances from 41% to 80% depending on the task. This approach can help to obtain information about how to improve harvesting process and reduce human workload in near future, which may be an important contribution for the search of sustainable agricultural practices.