基于深度卷积-递归神经网络的手绘草图识别方法

摘要

针对现有基于深度学习的手绘草图识别方法直接从整体上提取手绘草图的图像特征,而忽略了草图中笔画的顺序信息的问题,利用手绘草图的笔画顺序信息,将深度卷积神经网络与递归神经网络相结合,提出一种基于深度卷积-递归神经网络的手绘草图识别方法.首先按照绘画草图时的笔画顺序提取笔画,生成多幅子笔画草图,并形成一个笔画数依次递增的子笔画草图序列;然后采用深度卷积神经网络依次提取该序列中每一幅子笔画草图的图像特征,并将提取的图像特征按照原先子笔画草图排列的顺序进行排序,作为递归神经网络的输入;最后利用递归神经网络来构建不同图像特征间的时序关系,以提高手绘草图的识别准确率.在现有最大的手绘草图数据集TU-Berlin Sketch数据集上的实验结果表明,文中方法能有效地提升手绘草图的识别准确率.%The existing sketch recognition methods ignore the stroke order information in extracting the fea-ture of the sketch. This paper took the advantage of the stroke order information of the sketch and proposed a sketch recognition method based on deep convolutional-recurrent neural network, which combined the deep convolutional neural network and recurrent neural network. Firstly, the proposed method extracted the strokes of the sketch in sequence and obtained an ordered sequence of subsketches with increasing number of strokes. Secondly, a deep convolutional neural network was adapted to extract the feature of each sub-sketch in the ordered subsketch sequence and an ordered feature sequence was generated. Finally, the or-dered feature sequence was input into a modified recurrent neural network, which constructed the temporal relations among the different subsketches of the same sketch to improve the accuracy of the sketch recogni-tion. The experimental results on the largest freehand sketch dataset which is the TU-Berlin Sketch dataset show that the proposed method can effectively improve the recognition accuracy of freehand sketches.