Unified Architectural Approach to the Hybrid Mixed Challenge of Situational Assessment and Prediction. Task 1: Representing and Processing 3D Imagery.

摘要

The scientific goal of the full proposal focused on the development of a new cognitive architecture -- which has since been named Sigma -- that is based on graphical models, with a specific emphasis on the hybrid (combining continuous signal processing and discrete symbol processing) mixed (combining probabilistic representations of uncertainty with symbolic representations of knowledge) challenge of supporting robust situation assessment and prediction (SAP). Task 1, which was the one funded, specifically concerned the representation and processing of mental imagery in Sigma. The multi-year objectives of this task were to: (1) develop a means of representing mental imagery that leverages Sigma's unique capabilities and that is closely integrated with it (and that extends it to include (mixtures of) Gaussians for noisy continuous images); (2) implement mental imagery transformations -- such as translation, scaling and rotation -- within Sigma; and (3) produce predictions based on mental imagery, both in isolation and in conjunction with input about external reality. Except for the extension to Gaussians, these objectives were achieved, with 1D, 2D and 3D mental imagery grounded directly in the multidimensional piecewise-linear functions that are at the core of Sigma, and the standard imagery transformations modifying the locations of the boundaries between the regions of these functions. This combination surprisingly turned out to be general enough to support significant forms of processing that weren't originally conceived of as imagery, but which used numeric (metric) dimensions -- such as initializing, and returning results from, subgoals and processing rewards and value functions in reinforcement learning -- with the transformations turning out to directly yield a primitive form of mental arithmetic on these dimensions.