Building Innovative Speech Interfaces Using Patterns and Antipatterns of Commands for Controlling Loader Cranes

摘要

The paper describes research and development of innovative intelligent speech interaction systems between mobile lifting devices and their human operators, which use patterns and antipatterns of commands. A general processing scheme, using several functional modules, for the human-machine interactive communication has been presented, covering also the integration with vision and sensorial systems. The aim of the experimental research is to design a prototype of an innovative interaction system, equipped with a speech interface in a natural language, augmented reality and interactive manipulators with force feedback. The system is equipped with several adaptive intelligent layers for human biometric identification, speech recognition, word recognition, analysis and recognition of commands and messages, sentence meaning analysis, command effect analysis and safety assessment, process supervision and human reaction assessment. The paper also proposes a concept of synergistic control systems using patterns and antipatterns of commands. The concept consists of a methodology based on hybrid binary neural networks and deep learning convolutional networks.