ECCL: Explicit Correlation-Based Convolution Boundary Locator for Moment Localization

摘要

Moment localization in videos using natural language refers to finding the most relevant segment from the video with given a query in natural language form. In this paper, we present a new boundary-determining strategy called explicit correlation-based convolution boundary locator (ECCL), which can handle any lengths of videos and moments while leveraging fine-grained matching relationships. In this method, we first train a deep network to obtain the correlation scores between video clips and query statements. Subsequently, with the correlation scores, we utilize a convolution kernel to generate the boundary probability distribution. Finally, the start and end time indexes of the video moment are calculated with an optimization problem. Experiments on two publicly available datasets demonstrate the feasibility of ECCL.