Semantic scene classification for content-based image retrieval

Abstract

Content-based image indexing and retrieval have become important research problems with the use of large databases in a wide range of areas. Because of the constantly increasing complexity of the image content, low-level features are no longer sufficient for image content representation. In this study, a content-based image retrieval framework that is based on scene classification for image indexing is proposed. First, the images are segmented into regions by using their color and line structure information. By using the line structures of the images the regions that do not consist of uniform colors such as man made structures are captured. After all regions are clustered, each image is represented with the histogram of the region types it contains. Both multi-class and one-class classification models are used with these histograms to obtain the probability of observing different semantic classes in each image. Since a single class with the highest probability is not sufficient to model image content in an unconstrained data set with a large number of semantically overlapping classes, the obtained probability values are used as a new representation of the images and retrieval is performed on these new representations. In order to minimize the semantic gap, a relevance feedback approach that is based on the support vector data description is also incorporated. Experiments are performed on both Corel and TRECVID datasets and successful results are obtained.