Summarizing egocentric videos using deep features and optimal clustering

摘要

In this paper, we address the problem of summarizing egocentric videos using deep features and an optimal clustering approach. Based on an augmented pre-trained convolutional neural network (CNN), each frame in an egocentric video is represented by deep features. An optimal clustering algorithm, based on a center-surround model (CSM) and an Integer Knapsack type formulation (IK) for K-means, termed as CSMIK K-means, is applied next to obtain the summary. In the center surround model, we compute difference in entropy and the optical flow values between the central region and that of the surrounding region of each frame. In the integer knapsack formulation, each cluster is treated as an item whose cost is assigned from the center surround model. A potential set of clusters in CSMIK K-means is obtained from the chi-square distance between color histograms of successive frames. CSMIK K-Means evaluates different cluster formations and simultaneously determines the optimal number of clusters and the corresponding summary. Experimental evaluation on four well-known benchmark datasets clearly indicate the superiority of the proposed method over several state-of-the-art approaches. (C) 2020 Elsevier B.V. All rights reserved.