首页> 外文会议>ACM International Conference on Intelligent User Interfaces >Usage Patterns and Latent Semantic Analyses for Task Goal Inference of Multimodal User Interactions
【24h】

Usage Patterns and Latent Semantic Analyses for Task Goal Inference of Multimodal User Interactions

机译:用于多式联用户交互任务目标推断的使用模式和潜在语义分析

获取原文

摘要

This paper describes our work in usage pattern analysis and development of a latent semantic analysis framework for interpreting multimodal user input consisting speech and pen gestures. We have designed and collected a multimodal corpus of navigational inquiries. Each modality carries semantics related to domain-specific task goal. Each inquiry is annotated manually with a task goal based on the semantics. Multimodal input usually has a simpler syntactic structure than unimodal input and the order of semantic constituents is different in multimodal and unimodal inputs. Therefore, we proposed to use semantic analysis to derive the latent semantics from the multimodal inputs using latent semantic modeling (LSM). In order to achieve this, we parse the recognized Chinese spoken input for the spoken locative references (SLR). These SLRs are then aligned with their corresponding pen gesture(s). Then, we characterized the cross-modal integration pattern as 3-tuple multimodal terms with SLR, pen gesture type and their temporal relation. The inquiry-multimodal term matrix is then decomposed using singular value decomposition (SVD) to derive the latent semantics automatically. Task goal inference based on the latent semantics shows that the task goal inference accuracy on a disjoint test set is of 99%.
机译:本文介绍了我们在使用模式分析和开发中的工作,用于解释组成语音和笔手势的多模式用户输入的潜在语义分析框架。我们设计并收集了多峰的导航查询语料库。每个码形都携带与域特定的任务目标相关的语义。每个查询都以基于语义的任务目标手动注释。多模式输入通常具有比单峰输入更简单的句法结构,并且语义成分的顺序在多模式和单峰输入中不同。因此,我们建议使用语义分析来使用潜在语义建模(LSM)从多模式输入导出潜在语义。为了实现这一目标,我们解析了所公认的汉语口语输入,以获得口头定位参考(SLR)。然后将这些SLR与它们相应的笔手势对齐。然后,我们用SLR,PEN手势类型及其时间关系表征为3元组多模式术语的跨模型集成模式。然后使用奇异值分解(SVD)分解查询 - 多模式术语矩阵以自动导出潜在语义。基于潜在语义的任务目标推断表明,脱消测试集的任务目标推理准确性为99%。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号