Image understanding for global lifelog media cloud

摘要

We implemented media lifelog system with highlighting system with image analysis, video analysis and audio segmentation modules. Image analysis module has image classification, saliency region detection, face detection and facial expression recognition process. Video analysis module has cut detection and key frame detection process. And the result images of key frame detection is used as the input of image analysis module. Audio analysis module has audio segmentation process. ImageNet data is used for training and test database. The image classification accuracy is 83%. Automatic cut detection F1 score is 0.70. Cut detection F1 score is 0.80. Audio segmentation F score is 0.53. And facial expression recognition precision rate is 94.8% at 0.756 sec on a mobile phone.