首页> 外文会议>IEEE/CVF Conference on Computer Vision and Pattern Recognition >Semantic Video Segmentation by Gated Recurrent Flow Propagation
【24h】

Semantic Video Segmentation by Gated Recurrent Flow Propagation

机译:门控循环流传播的语义视频分割

获取原文

摘要

Semantic video segmentation is challenging due to the sheer amount of data that needs to be processed and labeled in order to construct accurate models. In this paper we present a deep, end-to-end trainable methodology for video segmentation that is capable of leveraging the information present in unlabeled data, besides sparsely labeled frames, in order to improve semantic estimates. Our model combines a convolutional architecture and a spatiotemporal transformer recurrent layer that is able to temporally propagate labeling information by means of optical flow, adaptively gated based on its locally estimated uncertainty. The flow, the recognition and the gated temporal propagation modules can be trained jointly, end-to-end. The temporal, gated recurrent flow propagation component of our model can be plugged into any static semantic segmentation architecture and turn it into a weakly supervised video processing one. Our experiments in the challenging CityScapes and Camvid datasets, and for multiple deep architectures, indicate that the resulting model can leverage unlabeled temporal frames, next to a labeled one, in order to improve both the video segmentation accuracy and the consistency of its temporal labeling, at no additional annotation cost and with little extra computation.
机译:语义视频分割具有挑战性,因为需要处理和标记大量数据才能构建准确的模型。在本文中,我们提出了一种深度,端到端的视频分割可训练方法,除了稀疏标记的帧外,该方法还可以利用未标记数据中存在的信息来改善语义估计。我们的模型结合了卷积架构和时空变压器循环层,该层能够通过光流在时间上传播标签信息,并根据其局部估计的不确定性进行自适应门控。流,识别和门控时间传播模块可以端到端联合训练。我们模型的时间门控循环流传播组件可以插入到任何静态语义分割架构中,然后将其转变为一个弱监督的视频处理架构。我们对具有挑战性的CityScapes和Camvid数据集进行了实验,并针对多种深度架构进行了实验,结果表明,该模型可以利用未标记的时间帧(紧贴标记的帧之后),以提高视频分割的准确性及其时间标记的一致性,无需额外的注释成本,几乎不需要额外的计算。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号