A novel and stable human detection and behavior recognition method based on depth sensor

摘要

To improve traditional video surveillance systems’ performance on human behavior recognition, a new system has been built. Not visual camera but depth camera is chosen as sensor. To adapt to the most common forward oblique view of camera, a normalized digital elevation map, whose pixel intensity indicates the elevation of the scene, is built from the depth image. Coordinates and intensity in the digital elevation map represent the position information about the scene. Oriented templates are proposed to match and detect the human head robustly in the elevation map. As for the excellent visibility of human head in the elevation map, we track the human head to get trajectory. By combining the trajectory information of human head with the elevation map, several predefined human behaviors are recognized. Our behavior recognition method is straightforward and robust. This uniqueness has no similar with the traditional machine learning and classification framework about human behavior recognition. Keywords depth map camera calibration digital elevation map human detection behavior recognition Page %P Close Plain text Look Inside Reference tools Export citation EndNote (.ENW) JabRef (.BIB) Mendeley (.BIB) Papers (.RIS) Zotero (.RIS) BibTeX (.BIB) Add to Papers Other actions Register for Journal Updates About This Journal Reprints and Permissions Share Share this content on Facebook Share this content on Twitter Share this content on LinkedIn Related Content Supplementary Material (0) References (36) References1.D. Meye, J. Denzler, H. Niemann (1998) Model based extraction of articulated objects in image sequences for gait analysis, IEEE International Conference on Image Processing 78–81.2.A. J. Lipton, H. F., R. S. Patil (1998) Moving target classification and tracking from real-time video, IEEE Workshop Application of Computer Vision 8–14.3.Y. L. Tian and A. Hampapur (2005) Robust Salient Motion Detection with Complex Background for Real-time Video Surveillance, IEEE Computer Society Workshop on Motion and Video Computing 30–35.4.R. Poppe (2010) A survey on vision-based human action recognition, Image and Vision Computing 28(6): 976–990.CrossRef5.J. Ding, M. L., K. Q. Huang and T. N. Tan (2010) Modeling Complex Scenes for Accurate Moving Objects Segmentation, 10th Asian conference on computer vision 592–604.6.N. Danal, B. T. (2005) Histograms of oriented gradients for human detection, Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition 886–893.7.P. Viola, M. J., D. Snow (2005) Detecting Pedestrians Using Patterns of Motion and Appearance, International Journal of Computer Vision 63(2): 153–161.CrossRef8.K. HOTTA (2002) Object Detection Method Based on Local Kernels and Automatic Kernel Selection by Kullback-Leibler Divergence, Proceeding of the Sixth IEEE Workshop on Applications of Computer Vision 105–111.9.I. P. Alonso, D. F. Llorca, M. Á. Sotelo, L. M. Bergasa, P. R. de Toro, J. Nuevo, M. Ocaña, M. Ángel, G. Garrido (2007) Combination of Feature Extraction Methods for SVM Pedestrian Detection, IEEE Transactions on Intelligent Transportation System 8(2):292–307.CrossRef10.L. Guo, L. Li, Y. Zhao, M. Zhang (2010) Study on Pedestrian Detection and Tracking with Monocular Vision, 2010 2nd International Conference on Computer Technology and Development 466–470.11.J. Ge, Y. Luo, and G. Tei (2009) Real-Time Pedestrian Detection and Tracking at Nighttime for Driver-Assistance Systems, IEEE Trans on Intelligent Transportation Systems 10(2): 283–298CrossRef12.C. Beleznai, B. Frühstück, H. Bischof (2006) Human Tracking by Fast Mean Shift Mode Seeking, Journal of Multimedia 1(1): 1–8.CrossRef13.L. Raskin, E. Rivlin, and M. Rudzsky (2008) Using Gaussian Process Annealing Particle Filter for 3D Human Tracking, EURASIP Journal on Advances in Signal Processing 1–13.14.R. Cucchiara, C. G., M. Piccardi, A. Prati, and S. Sirotti (2001) Improving Shadow Suppression in Moving Object Detection with HSV Color Information, Proc. Intelligent Transportation Systems Conference 334–339.15.W. Hu, X. Z., M. Hu, S. Maybank (2009) Occlusion Reasoning for Tracking Multiple People, IEEE Transactions on Circuits and Systems for Video Technology 19(1): 114–121.CrossRef16.Randolph Blake, Hugh Wilson (2011) Binocular vision, Vision Research 51(7): 754–770.CrossRef17.G. Gordon, X. Chen, R. Buck (2008) Person and Gesture Tracking with Smart Stereo Cameras, Proceedings of SPIE Vol. 6805: Three-Dimensional Image Capture and Applications.18.A. Shpunt, P. T. (2009) Optical design for zero order reduction, PrimeSense Ltd: America Pattern.19. http://​en.​wikipedia.​org/​wiki/​Kinect.20.K. Saenko, S. Karayev, Y. Jia, A. Shyr, A. Janoch, J. Long, M. Fritz, T. Darrell (2011) Practical 3-D Object Detection Using Category and Instance-level Appearance Models, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems 793–800.21.A. Kanezaki, T. Harada (2011) Scale and Rotation Invariant Color Features for Weakly-Supervised Object Learning in 3D Space, ICCV workshop 617–624.22.M. Luber, L. Spinello, K. O. Arras (2011) People Tracking in RGB-D Data With On-line Boosted Target Models, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems 3844–3849.23.T. Nakamura (2011) Real-time 3-D Object Tracking Using Kinect Sensor, Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics 784–788.CrossRef24.L. Spinello, K. O. Arras (2011) People Detection in RGB-D Data, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems 3838–3843.CrossRef25.T. Gill, J. M. Keller, D. T. Anderson, R. H. Luke III (2011) A system for change detection and human recognition in voxel space using the Microsoft Kinect sensor, Proceedings of the 2011 IEEE International Conference on Applied Imagery Pattern Recognition Workshop (AIPR) 1–8.CrossRef26.H. Hu, B. Li, B. Huang, Y. Cui (2011) Interaction System of Tread mill Games based on Depth Maps and CAM-Shift, 2011 IEEE 3rd International Conference on Communication Software and Networks (ICCSN) 219–222.27.Y. Qi, K. Suzuki, H. Wu and Q. Chen (2011) EK-means Tracker: A Pixel-wise Tracking Algorithm using Kinect, Third Chinese Conference on Intelligent Visual Surveillance 77–80.28.A. Bevilacqua, L. D. Stefano, P. Azzari (2006) People tracking using a Time-of-Flight depth sensor, Proceedings of the IEEE International Conference on Video and Signal Based Surveillance (AVSS’06) 89–93CrossRef29.R. Tanner, M. Studer, A. Zanoli (2008) Andreas Hartmann, People Detection and Tracking with TOF Sensor, AVSS’08 356–361.30.Y. I. Abdel-Aziz and H. M. Karara (1971) Direct linear transformation into object space coordinates in close-range photogrammetry, Proc. Symp. Close-Range Photogrammetry 1–18.31.R. Tsai (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-theshelf TV cameras and lenses, IEEE Journal of Robotics and Automation, 3(4): 323–344.CrossRef32.Z. Zhang (2000) A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11): 1330–1334.CrossRef33.P. Rakprayoon, M. Ruchanurucks, and A. Coundoul (2011) Kinect-based Obstacle Detection for Manipulator, IEEE/SICE International Symposium on System Integration (SII) 68–73CrossRef34.J. Smisek, M. Jancosek and T. Pajdlaj (2011) 3D with Kinect, IEEE International Conference on Computer Vision Workshops 1154–1160.35.V. Frati, D. Prattichizzo (2011) Using Kinect for hand tracking and rendering in wearable haptics, IEEE World Haptics Conference 317–321.CrossRef36.F. A. Kondori, Sh. Yousefi, H. Li, S. Sonning, S. Sonning (2011) 3D Head Pose Estimation Using the Kinect, 2011 International Conference on Wireless Communications and Signal Processing (WCSP) 1–4.CrossRef About this Article Title A novel and stable human detection and behavior recognition method based on depth sensor Journal 3D Research 4:3 Online DateMay 2013 DOI 10.1007/3DRes.02(2013)3 Online ISSN 2092-6731 Publisher 3D Display Research Center Additional Links Register for Journal Updates Editorial Board About This Journal Manuscript Submission Topics Signal, Image and Speech Processing Computer Imaging, Vision, Pattern Recognition and Graphics Optics, Optoelectronics, Plasmonics and Optical Devices Keywords depth map camera calibration digital elevation map human detection behavior recognition Industry Sectors Electronics Engineering IT & Software Telecommunications Authors Shuqiang Yang (191) Biao Li (191) Author Affiliations 191. ATR laboratory at National University of Defense Technology, Changsha, Hunan Province, China Continue reading... To view the rest of this content please follow the download PDF link above.