Load balanced locality-aware parallel SGD on multicore architectures for latent factor based collaborative filtering

Date
2023-04-20
Editor(s)
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Future Generation Computer Systems
Print ISSN
0167-739X
Electronic ISSN
1872-7115
Publisher
Elsevier BV * North-Holland
Volume
146
Issue
Pages
207 - 221
Language
en
Journal Title
Journal ISSN
Volume Title
Series
Abstract

We investigate the parallelization of Stochastic Gradient Descent (SGD) for matrix completion on multicore architectures. We provide an experimental analysis of current SGD algorithms to find out their bottlenecks and limitations. Grid-based methods suffer from load imbalance among 2D blocks of the rating matrix, especially when datasets are skewed and sparse. Asynchronous methods, on the other hand, can face cache issues due to their memory access pattern. We propose bin-packing-based block balancing methods that are alternative to the recently proposed BaPa method. We then introduce Locality Aware SGD (LASGD), a grid-based asynchronous parallel SGD algorithm that efficiently utilizes cache by changing nonzero update sequence without affecting factor update order and carefully arranging latent factor matrices in the memory. Combined with our proposed load balancing methods, our experiments show that LASGD performs significantly better than alternative approaches in parallel shared-memory systems.

Course
Other identifiers
Book Title
Citation
Published Version (Please cite this version)