Automatic semantic indexing in multimedia information retrieval.

摘要

In multimedia information retrieval, low-level features can be extracted automatically, but it is very hard to derive the high-level semantic concepts from the low-level features. The problem is called "semantic gap". Automatic semantic indexing is an important research field in that it tries to address the semantic gap in an automatic, effective and efficient manner. On the other hand, how to properly evaluate the effectiveness of the low-level features also remains a challenge and an important task.; This dissertation presents two novel approaches addressing the automatic semantic indexing problem, as well as new approaches for evaluating the low-level features.; In the first semantic indexing approach, we use color-texture classification to generate a novel semantic codebook, which is then used to extract and index image regions. The content of a region depicts the semantic description derived from the lower-level features of the region. The context of regions in an image represents their relationships in the image. The semantic codebook provides a way of automatically deriving the content and context of image regions. The experimental results demonstrate that the approach outperforms the traditional image retrieval approaches.; In the second approach, we have tackled the semantic indexing problem by mining the decisive feature patterns. Intuitively, a decisive feature pattern is a combination of feature values that are unique and significant for describing a semantic concept. Interesting algorithms are developed and analyzed to mine the decisive feature patterns and construct the semantic rule bases, to automatically recognize semantic concepts in images. A systematic performance study on large image databases shows that our method has good potentials in addressing the major problems of the automatic semantic indexing.; Finally, a general mathematical model is proposed to measure low-level features' contributions to the image content, for evaluating the low-level features. As an illustration of evaluation, the contributions of color and area to the image content are measured to evaluate the performance of the Color Histogram and the Color Coherence Vectors methods.