Fast optimal load balancing algorithms for 1D partitioning

Date
2004
Advisor
Instructor
Source Title
Journal of Parallel and Distributed Computing
Print ISSN
0743-7315
1096-0848
Electronic ISSN
Publisher
Academic Press
Volume
64
Issue
8
Pages
974 - 996
Language
English
Type
Article
Journal Title
Journal ISSN
Volume Title
Abstract

The one-dimensional decomposition of nonuniform workload arrays with optimal load balancing is investigated. The problem has been studied in the literature as the "chains-on-chains partitioning" problem. Despite the rich literature on exact algorithms, heuristics are still used in parallel computing community with the "hope" of good decompositions and the "myth" of exact algorithms being hard to implement and not runtime efficient. We show that exact algorithms yield significant improvements in load balance over heuristics with negligible overhead. Detailed pseudocodes of the proposed algorithms are provided for reproducibility. We start with a literature review and propose improvements and efficient implementation tips for these algorithms. We also introduce novel algorithms that are asymptotically and runtime efficient. Our experiments on sparse matrix and direct volume rendering datasets verify that balance can be significantly improved by using exact algorithms. The proposed exact algorithms are 100 times faster than a single sparse-matrix vector multiplication for 64-way decompositions on the average. We conclude that exact algorithms with proposed efficient implementations can effectively replace heuristics. © 2004 Elsevier Inc. All rights reserved.

Course
Other identifiers
Book Title
Keywords
Chains-on-chains partitioning, Dynamic programming, Image-space parallel volume rendering, Iterative refinement, One-dimensional partitioning, Optimal load balancing, Parallel sparse matrix vector multiplication, Parametric search, Algorithms, Dynamic programming, Graph theory, Heuristic methods, Matrix algebra, Vectors, One-dimensional partitioning, Optimal load balancing, Parallel sparse matrix vector multiplication, Parallel processing systems
Citation
Published Version (Please cite this version)