Simultaneous input and output matrix partitioning for outer-product-parallel sparse matrix-matrix multiplication
SIAM Journal on Scientific Computing
Society for Industrial and Applied Mathematics
C568 - C590
Item Usage Stats
FFor outer-product-parallel sparse matrix-matrix multiplication (SpGEMM) of the form C=A×B, we propose three hypergraph models that achieve simultaneous partitioning of input and output matrices without any replication of input data. All three hypergraph models perform conformable one-dimensional (1D) columnwise and 1D rowwise partitioning of the input matrices A and B, respectively. The first hypergraph model performs two-dimensional (2D) nonzero-based partitioning of the output matrix, whereas the second and third models perform 1D rowwise and 1D columnwise partitioning of the output matrix, respectively. This partitioning scheme induces a two-phase parallel SpGEMM algorithm, where communication-free local SpGEMM computations constitute the first phase and the multiple single-node-accumulation operations on the local SpGEMM results constitute the second phase. In these models, the two partitioning constraints defined on weights of vertices encode balancing computational loads of processors during the two separate phases of the parallel SpGEMM algorithm. The partitioning objective of minimizing the cutsize defined over the cut nets encodes minimizing the total volume of communication that will occur during the second phase of the parallel SpGEMM algorithm. An MPI-based parallel SpGEMM library is developed to verify the validity of our models in practice. Parallel runs of the library for a wide range of realistic SpGEMM instances on two large-scale parallel systems JUQUEEN (an IBM BlueGene/Q system) and SuperMUC (an Intel-based cluster) show that the proposed hypergraph models attain high speedup values. © 2014 Society for Industrial and Applied Mathematics.
Sparse matrix-matrix multiplication
Published Version (Please cite this version)http://dx.doi.org/10.1137/13092589X
Showing items related by title, author, creator and subject.
Encapsulating multiple communication-cost metrics in partitioning sparse rectangular matrices for parallel matrix-vector multiplies Uçar, B.; Aykanat, Cevdet (SIAM, 2004)This paper addresses the problem of one-dimensional partitioning of structurally unsymmetric square and rectangular sparse matrices for parallel matrix-vector and matrix-transpose-vector multiplies. The objective is to ...
Locality-aware parallel sparse matrix-vector and matrix-transpose-vector multiplication on many-core processors Karsavuran, M. O.; Akbudak K.; Aykanat, Cevdet (Institute of Electrical and Electronics Engineers, 2016)Sparse matrix-vector and matrix-transpose-vector multiplication (SpMMTV) repeatedly performed as z ← ATx and y ← A z (or y ← A w) for the same sparse matrix A is a kernel operation widely used in various iterative solvers. ...
Matrix-geometric solutions of M/G/1-type Markov chains: A unifying generalized state-space approach Akar, N.; Oǧuz, N.C.; Sohraby, K. (1998)In this paper, we present an algorithmic approach to find the stationary probability distribution of M/G/1-type Markov chains which arise frequently in performance analysis of computer and communication networ ks. The ...