Scene classification using bag-of-regions representation

Abstract

Significant growth of multimedia data creates the need for more complicated approaches in image understanding, classification and retrieval. Semantic scene classification is a popular research area which categorizes images into semantic categories for applications like content based image retrieval. In the near future, content based image retrieval will be much more important especially for the next generation internet technologies so new approaches are very welcomed in this subject. Research has showed that classifying images using components like regions, pixels or objects is a challenging work because of the ambiguity of the visual data. The main idea about image classification is to find similarities between these components to get information about the content of the image. This thesis describes our work on classification of outdoor scenes. As the first step, regions are extracted using one-class classification and patch-based clustering algorithms. The components (pixels, regions and objects) in outdoor images have particular spatial and geometric interactions so dividing images into meaningfully clustered regions has important benefits for a detailed content analysis. For region clustering, features from different levels make specific contributions but to avoid the ambiguity, we need to use low level information and more global information together for the clustering step. Also, using spatial relationships between clustered regions, we can make inference about the detailed content of outdoor images from specific to general. Therefore, after rough segmentation, scene representations are constructed with and without spatial information. At the final step Bayesian classification approach is used with the two different scene representations. The developed methods were tested on the MIT LabelMe dataset, and the results showed that using regions and their spatial relationships improved the classification accuracy.