Natural Anticipation and Selection of Attention within Sustainable Intelligent Multimodal Systems by Collective Intelligent Agents

摘要

An ambient intelligent environment is definitely a prerequisite for anticipating human's needs and catching the attention of humans and systems and e.g. notifying them of possibly hazardous or lucrative situations. But how to endow such an environment with natural anticipatory and attentive features is still a hardly ever properly addressed question. First, we give from a cognitive scientific perspective an account for natural anticipation and selection of attention (NASA). We describe why, when and how exploratory and goal-directed acts by humans could be controlled while optimizing the changing and limited structural and functional capabilities of multimodal sensor, cognitive and actuator systems. Second, we make explicit how NASA is embedded and embodied in sustainable intelligent multimodal systems (SIMS) such that humans and systems can interact taking their own and environmental contexts into account. Thereto, we present a mathematical-physical framework for modeling such systems. Systems, which are modeled this way, robustly and reliably balance multi-sensor detection and fusion; multimodal fusion, dialogue planning and fission; multi actuator fission, rendering and presentation schemes to improve human-human, human-system and system-system interactions. The NASA (pre-) schemes are active within every phase of perception-decision-action cycles and are gauged and renormalized to internal and environmental physics (a pre-scheme is defined as a hidden and latent physical structure and organization). After determining and assessing appropriate fitness measures and utility measures for those schemes they can be embedded and embodied by reinforcement learning and self-organization. Finally, collective intelligent agents (CIA) are proposed to distribute, store, extend, maintain, optimize, diversify and therewith sustain the NASA (pre-) schemes within SIMS for both humans and systems.