Efficiency and effectiveness of query processing in cluster-based retrieval

Date
2004
Authors
Can, F.
Altingövde I.S.
Demir, E.
Editor(s)
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Information Systems
Print ISSN
0306-4379
1873-6076
Electronic ISSN
Publisher
Elsevier
Volume
29
Issue
8
Pages
697 - 717
Language
English
Journal Title
Journal ISSN
Volume Title
Series
Abstract

Our research shows that for large databases, without considerable additional storage overhead, cluster-based retrieval (CBR) can compete with the time efficiency and effectiveness of the inverted index-based full search (FS). The proposed CBR method employs a storage structure that blends the cluster membership information into the inverted file posting lists. This approach significantly reduces the cost of similarity calculations for document ranking during query processing and improves efficiency. For example, in terms of in-memory computations, our new approach can reduce query processing time to 39% of FS. The experiments confirm that the approach is scalable and system performance improves with increasing database size. In the experiments, we use the cover coefficient-based clustering methodology (C3M), and the Financial Times database of TREC containing 210158 documents of size 564 MB defined by 229748 terms with total of 29545234 inverted index elements. This study provides CBR efficiency and effectiveness experiments using the largest corpus in an environment that employs no user interaction or user behavior assumption for clustering. © 2003 Elsevier Ltd. All rights reserved.

Course
Other identifiers
Book Title
Citation
Published Version (Please cite this version)