Parallel stochastic gradient descent with sub-iterations on distributed memory systems
| buir.advisor | Özdal, M. Mustafa | |
| dc.contributor.author | Çağlayan, Orhun | |
| dc.date.accessioned | 2022-03-22T06:11:04Z | |
| dc.date.available | 2022-03-22T06:11:04Z | |
| dc.date.copyright | 2022-02 | |
| dc.date.issued | 2022-02 | |
| dc.date.submitted | 2022-03-04 | |
| dc.department | Department of Computer Engineering | en_US |
| dc.description | Cataloged from PDF version of article. | en_US |
| dc.description | Thesis (Master's): Bilkent University, Department of Computer Engineering, İhsan Doğramacı Bilkent University, 2022. | en_US |
| dc.description | Includes bibliographical references (leaves 41-43). | en_US |
| dc.description.abstract | We investigate parallelization of the stochastic gradient descent (SGD) algorithm for solving the matrix completion problem. Applications in the literature show that stale data usage and communication costs are important concerns that affect the performance of parallel SGD applications. We first briefly visit the stochastic gradient descent algorithm and matrix partitioning for parallel SGD. Then we define the stale data problem and communication costs. In order to improve the performance of parallel SGD, we propose a new algorithm with intra-iteration synchronization (referred as sub-iterations) to decrease communication costs and stale data usage. Experimental results show that using sub-iterations can de-crease staleness up to 95% and communication volume up to 47%. Furthermore, using sub-iterations can improve test error up to 60% when compared to the conventional parallel SGD implementation that does not use sub-iterations. | en_US |
| dc.description.degree | M.S. | en_US |
| dc.description.statementofresponsibility | by Orhun Çağlayan | en_US |
| dc.embargo.release | 2022-09-04 | |
| dc.format.extent | x, 43 leaves : charts ; 30 cm. | en_US |
| dc.identifier.itemid | B160829 | |
| dc.identifier.uri | http://hdl.handle.net/11693/77841 | |
| dc.language.iso | English | en_US |
| dc.publisher | Bilkent University | en_US |
| dc.rights | info:eu-repo/semantics/openAccess | en_US |
| dc.subject | Stochastic gradient descent | en_US |
| dc.subject | Matrix completion | en_US |
| dc.subject | Parallel computing | en_US |
| dc.subject | Distributed memory systems | en_US |
| dc.title | Parallel stochastic gradient descent with sub-iterations on distributed memory systems | en_US |
| dc.title.alternative | Dağıtık bellekli sistemlerde alt-iterasyonlu paralel olasılıksal gradyan alçalma | en_US |
| dc.type | Thesis | en_US |