The aim of this paper is to present a novel system which tracks the motion of a drummer and generates the corresponding drum sounds. The input video sequence from a camera is processed in real-time by using local and adaptive color segmentation and Kalman filter based tracking. The Kalman filter is used to predict the "hits" so that we can overcome the processing delays and provide a more-realistic drumming experience. We use a local and adaptive search to detect the effective points of the drum sticks, which ensures robustness to background clutter and reduces the computational burden. We have developed a working demo and evaluated its performance by comparing with the output signal of an electronic drum pad. We observed that the timing errors have an average of −8.4 ms and a standard deviation of 5.4 ms in a real drumming experiment consisting of 121 hits.
展开▼
