Interactive music playback system utilizing gestures

摘要

An interactive music system in accordance with various aspects of the invention lets a user control the playback of recorded music according to gestures entered via an input device, such as a mouse. The system includes modules which interpret input gestures made on a computer input device and adjust the playback of audio data in accordance with input gesture data. Various methods for encoding sound information in an audio data produce with meta-data indicating how it can be varied during playback are also disclosed. More specifically, a gesture input system receives user input from a device, such as a mouse, and interprets this data as one of a number of predefined gestures which are assigned an emotional or interpretive meaning according to a “character” hierarchy or library of gesture descriptions. The received gesture inputs are used to alter the character of music which is being played in accordance with the meaning of the gesture. For example, an excited gesture can effect the playback in one way, while a quiet playback may affect it in another. The specific result is a combination of the gesture made by the user, its interpretation by the computer, and a determination of how the interpreted gesture should effect the playback. Entry of a excited gesture thus may brighten the playback, e.g., by changing increasing the tempo, changing from a minor to major key, varying the instruments used or the style in which they are played, etc. In addition, the effects can be cumulative, allowing a user to progressively alter the playback. To further enhance the interactive nature of the system, users can be given the ability to alter the effect of a given gesture or assign a gesture to specific places in the character hierarchy.