Partitioning functions for steteful data parallelism in stream processing

Simple item page
dc.citation.epage539en_US
dc.citation.issueNumber4en_US
dc.citation.spage517en_US
dc.citation.volumeNumber23en_US
dc.contributor.authorGedik, B.en_US
dc.date.accessioned2015-07-28T12:04:06Z
dc.date.available2015-07-28T12:04:06Z
dc.date.issued2014en_US
dc.departmentDepartment of Computer Engineeringen_US
dc.description.abstractIn this paper we study partitioning functions for stream processing systems that employ stateful data parallelism to improve application throughput. In particular, we develop partitioning functions that are effective under workloads where the domain of the partitioning key is large and its value distribution is skewed. We define various desirable properties for partitioning functions, ranging from balance properties such as memory, processing, and communication balance, structural properties such as compactness and fast lookup, and adaptation properties such as fast computation and minimal migration. We introduce a partitioning function structure that is compact and develop several associated heuristic construction techniques that exhibit good balance and low migration cost under skewed workloads. We provide experimental results that compare our partitioning functions to more traditional approaches such as uniform and consistent hashing, under different workload and application characteristics, and show superior performance.en_US
dc.description.provenanceMade available in DSpace on 2015-07-28T12:04:06Z (GMT). No. of bitstreams: 1 10.1007-s00778-013-0335-9.pdf: 2997623 bytes, checksum: d383d88bc24bd009c428974d009647a1 (MD5)en
dc.identifier.doi10.1007/s00778-013-0335-9en_US
dc.identifier.issn1066-8888
dc.identifier.urihttp://hdl.handle.net/11693/12957
dc.language.isoEnglishen_US
dc.publisherAssociation for Computing Machineryen_US
dc.relation.isversionofhttp://dx.doi.org/10.1007/s00778-013-0335-9en_US
dc.source.titleThe VLDB Journalen_US
dc.subjectStream processingen_US
dc.subjectLoad balanceen_US
dc.subjectPartitioning functionsen_US
dc.titlePartitioning functions for steteful data parallelism in stream processingen_US
dc.typeArticleen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
10.1007-s00778-013-0335-9.pdf
Size:
2.86 MB
Format:
Adobe Portable Document Format
Download

Collections

Scholarly Publications - Computer Engineering