Personal Orchestra: conducting audio/video music recordings

摘要

Personal Orchestra lets anybody conduct an electronic orchestra at a new level of realism: users interact not with a synthetic, but an original audio and video recording of a real orchestra - the Vienna Philharmonic. Nevertheless, they can interactively control not only volume and instrumentation, but also tempo of the orchestra, using natural conducting gestures. A gesture tracking and predicting algorithm interprets user input, and a high-fidelity playback algorithm renders audio and video at variable speed without time-stretching artifacts such as pitch changes. The system was designed following a set of user interface design patterns for interactive exhibits. It is being used as the "Virtual Conductor" exhibit in the House of Music Vienna by hundreds of visitors every day.