A Deformable Interface for Human Touch Recognition Using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks

摘要

User interfaces provide an interactive window between physical and virtualenvironments. A new concept in the field of human-computer interaction is asoft user interface; a compliant surface that facilitates touch interactionthrough deformation. Despite the potential of these interfaces, they currentlylack a signal processing framework that can efficiently extract informationfrom their deformation. Here we present OrbTouch, a device that usesstatistical learning algorithms, based on convolutional neural networks, to mapdeformations from human touch to categorical labels (i.e., gestures) and touchlocation using stretchable capacitor signals as inputs. We demonstrate thisapproach by using the device to control the popular game Tetris. OrbTouchprovides a modular, robust framework to interpret deformation in soft media,laying a foundation for new modes of human computer interaction through shapechanging solids.