This paper addresses the problem of detecting and identifying persons with a mobile robot, by sensory fusion of thermal and colour vision information. In the proposed system, people are first detected with a thermal camera, using image analysis techniques to segment the persons in the thermal images. This information is then used to segment the corresponding regions of the colour images, using an affine transformation to solve the image correspondence between the two cameras. After segmentation, the region of the image containing a person is further divided into regions corresponding to the person's head, torso and legs. Temperature and colour features are then extracted from each region for input to a pattern recognition system. Three alternative classification methods were investigated in experiments with a moving mobile robot and moving persons in an office environment. The best identification performance was obtained with a dynamic recognition method based on a Bayes classifier, which takes into account evidence accumulated in a sequence of images.
展开▼
