首页> 外文期刊>IEEE transactions on visualization and computer graphics >A Semantic-Based Method for Visualizing Large Image Collections
【24h】

A Semantic-Based Method for Visualizing Large Image Collections

机译:基于语义的大图像集合可视化方法

获取原文
获取原文并翻译 | 示例

摘要

Interactive visualization of large image collections is important and useful in many applications, such as personal album management and user profiling on images. However, most prior studies focus on using low-level visual features of images, such as texture and color histogram, to create visualizations without considering the more important semantic information embedded in images. This paper proposes a novel visual analytic system to analyze images in a semantic-aware manner. The system mainly comprises two components: a semantic information extractor and a visual layout generator. The semantic information extractor employs an image captioning technique based on convolutional neural network (CNN) to produce descriptive captions for images, which can be transformed into semantic keywords. The layout generator employs a novel co-embedding model to project images and the associated semantic keywords to the same 2D space. Inspired by the galaxy metaphor, we further turn the projected 2D space to a galaxy visualization of images, in which semantic keywords and images are visually encoded as stars and planets. Our system naturally supports multi-scale visualization and navigation, in which users can immediately see a semantic overview of an image collection and drill down for detailed inspection of a certain group of images. Users can iteratively refine the visual layout by integrating their domain knowledge into the co-embedding process. Two task-based evaluations are conducted to demonstrate the effectiveness of our system.
机译:大型图像集合的交互式可视化在许多应用程序中都很重要且很有用,例如个人相册管理和图像用户配置文件。但是,大多数先前的研究集中在使用图像的低级视觉特征(例如纹理和颜色直方图)来创建可视化效果,而无需考虑图像中嵌入的更重要的语义信息。本文提出了一种新颖的视觉分析系统,以语义感知的方式分析图像。该系统主要包括两个组件:语义信息提取器和视觉布局生成器。语义信息提取器采用基于卷积神经网络(CNN)的图像字幕技术为图像生成描述性字幕,可以将其转换为语义关键字。布局生成器采用新颖的共嵌入模型将图像和关联的语义关键字投影到同一2D空间。受星系隐喻的启发,我们将投影的2D空间进一步转变为图像的星系可视化,其中语义关键字和图像在视觉上被编码为恒星和行星。我们的系统自然支持多尺度的可视化和导航,其中用户可以立即查看图像集合的语义概述,并向下钻取以查看特定图像组的详细信息。用户可以通过将他们的领域知识集成到共嵌入过程中来迭代地优化视觉布局。进行了两次基于任务的评估,以证明我们系统的有效性。

著录项

  • 来源
  • 作者单位

    Zhejiang Univ, State Key Lab CAD&CG, Hangzhou 310027, Zhejiang, Peoples R China|Alibaba Zhejiang Univ, Joint Inst Frontier Technol, Hangzhou 310027, Zhejiang, Peoples R China;

    Zhejiang Univ, State Key Lab CAD&CG, Hangzhou 310027, Zhejiang, Peoples R China;

    Zhejiang Univ, State Key Lab CAD&CG, Hangzhou 310027, Zhejiang, Peoples R China;

    Tongji Univ, Coll Design & Innovat, Shanghai 200092, Peoples R China;

    Zhejiang Univ, State Key Lab CAD&CG, Hangzhou 310027, Zhejiang, Peoples R China|Alibaba Zhejiang Univ, Joint Inst Frontier Technol, Hangzhou 310027, Zhejiang, Peoples R China;

  • 收录信息
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

    Image visualization; semantic layout; CNN; image captioning;

    机译:图像可视化;语义布局;CNN;图像标题;

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号