Site-based partitioning and repartitioning techniques for parallel pagerank computation

Date
2011-05
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
IEEE Transactions on Parallel and Distributed Systems
Print ISSN
1045-9219
Electronic ISSN
1558-2183
Publisher
Institute of Electrical and Electronics Engineers
Volume
22
Issue
5
Pages
786 - 802
Language
English
Type
Article
Journal Title
Journal ISSN
Volume Title
Series
Abstract

The PageRank algorithm is an important component in effective web search. At the core of this algorithm are repeated sparse matrix-vector multiplications where the involved web matrices grow in parallel with the growth of the web and are stored in a distributed manner due to space limitations. Hence, the PageRank computation, which is frequently repeated, must be performed in parallel with high-efficiency and low-preprocessing overhead while considering the initial distributed nature of the web matrices. Our contributions in this work are twofold. We first investigate the application of state-of-the-art sparse matrix partitioning models in order to attain high efficiency in parallel PageRank computations with a particular focus on reducing the preprocessing overhead they introduce. For this purpose, we evaluate two different compression schemes on the web matrix using the site information inherently available in links. Second, we consider the more realistic scenario of starting with an initially distributed data and extend our algorithms to cover the repartitioning of such data for efficient PageRank computation. We report performance results using our parallelization of a state-of-the-art PageRank algorithm on two different PC clusters with 40 and 64 processors. Experiments show that the proposed techniques achieve considerably high speedups while incurring a preprocessing overhead of several iterations (for some instances even less than a single iteration) of the underlying sequential PageRank algorithm. © 2011 IEEE.

Course
Other identifiers
Book Title
Keywords
Graph partitioning, Hypergraph partitioning, PageRank, Parallelization, Repartitioning, Sparse matrix partitioning, Sparse matrix - vector multiplication, Web search
Citation
Published Version (Please cite this version)