Feature-point extraction is a fundamental step in many applications, such as image matching and Simultaneous Localization and Mapping (SLAM). The CNN-based feature-point extraction methods have made significant signs of progress in both feature-point detection and descriptor generation compared with handcrafted processes. However, the computational and storage complexity makes it difficult for CNN to run on real-time embedded systems. In this paper, we aim to deploy the advanced CNN-based feature-point extraction methods onto real-time embedded FPGA systems. We optimize the softmax data flow so that the computation of softmax and NMS can be reduced by $64 imes$. We generate the normalized descriptors after picking the feature-points with the highest confidence so that the computation cost of normalization is reduced by $1500 imes$. We use fixed-point in both of the CNN backbone and the post-processing operations, and implement them on the ZCU102 FPGA platform. The experimental results show that our proposed hardware-software co-design CNN-based feature-point extraction method outperforms the handcrafted techniques. Our feature-point extraction on the embedded platform runs at the speed of 20 fps, meeting the real-time requirement.
展开▼
机译：特征点提取是许多应用程序中的基本步骤，例如图像匹配以及同时定位和映射（SLAM）。与手工处理相比，基于CNN的特征点提取方法在特征点检测和描述符生成方面都取得了明显的进步迹象。但是，计算和存储复杂性使CNN难以在实时嵌入式系统上运行。在本文中，我们旨在将基于CNN的高级特征点提取方法部署到实时嵌入式FPGA系统上。我们优化softmax数据流，以使softmax和NMS的计算可以减少$ 64 \ times $。我们在选择具有最高置信度的特征点之后生成归一化描述符，从而将归一化的计算成本降低了1500 \ times $。我们在CNN主干和后处理操作中都使用定点，并在ZCU102 FPGA平台上实现它们。实验结果表明，本文提出的基于CNN的软硬件协同设计特征点提取方法优于手工技术。我们在嵌入式平台上的特征点提取以20 fps的速度运行，满足实时需求。
展开▼
