Minimizing communication through computational redundancy in parallel iterative solvers
Author
Torun, Fahreddin Şükrü
Advisor
Aykanat, Cevdet
Date
2011Publisher
Bilkent University
Language
English
Type
ThesisItem Usage Stats
74
views
views
47
downloads
downloads
Abstract
Sparse matrix vector multiplication (SpMxV) of the form y = Ax is a kernel
operation in iterative linear solvers used in scientific applications. In these
solvers, the SpMxV operation is performed repeatedly with the same sparse matrix
through iterations until convergence. Depending on the matrix and its decomposition,
parallel SpMxV operation necessitates communication among processors
in the parallel environment. The communication can be reduced by intelligent
decomposition. However, we can further decrease the communication through
data replication and redundant computation. The communication occurs due to
the transfer of x-vector entries in row-parallel SpMxV computation. The input
vector x of the next iteration is computed from the output vector of the current
iteration through linear vector operations. Hence, a processor may compute a
y-vector entry redundantly, which leads to a x-vector entry in the following iteration,
instead of receiving that x-vector entry from another processor. Thus,
redundant computation of that y-vector entry may lead to reduction in communication.
In this thesis, we devise a directed-graph-based model that correctly captures
the computation and communication pattern for above-mentioned iterative
solvers. Moreover, we formulate the communication minimization by utilizing
redundant computation of y-vector entries as a combinatorial problem on this
directed graph model. We propose two heuristics to solve this combinatorial
problem. Experimental results indicate that the communication reducing strategy
by redundantly computing is promising.