Accelerating the HyperLogLog cardinality estimation algorithm

Date
2017
Authors
Bozkus, C.
Fraguela, B. B.
Editor(s)
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Scientific Programming
Print ISSN
1058-9244
Electronic ISSN
Publisher
Hindawi Limited
Volume
2017
Issue
Pages
Language
English
Journal Title
Journal ISSN
Volume Title
Series
Abstract

In recent years, vast amounts of data of different kinds, from pictures and videos from our cameras to software logs from sensor networks and Internet routers operating day and night, are being generated. This has led to new big data problems, which require new algorithms to handle these large volumes of data and as a result are very computationally demanding because of the volumes to process. In this paper, we parallelize one of these new algorithms, namely, the HyperLogLog algorithm, which estimates the number of different items in a large data set with minimal memory usage, as it lowers the typical memory usage of this type of calculation from O(n) to O(1). We have implemented parallelizations based on OpenMP and OpenCL and evaluated them in a standard multicore system, an Intel Xeon Phi, and two GPUs from different vendors. The results obtained in our experiments, in which we reach a speedup of 88.6 with respect to an optimized sequential implementation, are very positive, particularly taking into account the need to run this kind of algorithm on large amounts of data. © 2017 Cem Bozkus and Basilio B. Fraguela.

Course
Other identifiers
Book Title
Citation
Published Version (Please cite this version)