Elastic scaling for data stream processing

dc.citation.epage1463en_US
dc.citation.issueNumber6en_US
dc.citation.spage1447en_US
dc.citation.volumeNumber25en_US
dc.contributor.authorGedik, B.en_US
dc.contributor.authorSchneider S.en_US
dc.contributor.authorHirzel M.en_US
dc.contributor.authorWu, Kun-Lungen_US
dc.date.accessioned2016-02-08T11:03:15Z
dc.date.available2016-02-08T11:03:15Z
dc.date.issued2014en_US
dc.departmentDepartment of Computer Engineeringen_US
dc.description.abstractThis article addresses the profitability problem associated with auto-parallelization of general-purpose distributed data stream processing applications. Auto-parallelization involves locating regions in the application's data flow graph that can be replicated at run-time to apply data partitioning, in order to achieve scale. In order to make auto-parallelization effective in practice, the profitability question needs to be answered: How many parallel channels provide the best throughput? The answer to this question changes depending on the workload dynamics and resource availability at run-time. In this article, we propose an elastic auto-parallelization solution that can dynamically adjust the number of channels used to achieve high throughput without unnecessarily wasting resources. Most importantly, our solution can handle partitioned stateful operators via run-time state migration, which is fully transparent to the application developers. We provide an implementation and evaluation of the system on an industrial-strength data stream processing platform to validate our solution. © 1990-2012 IEEE.en_US
dc.description.provenanceMade available in DSpace on 2016-02-08T11:03:15Z (GMT). No. of bitstreams: 1 bilkent-research-paper.pdf: 70227 bytes, checksum: 26e812c6f5156f83f0e77b261a471b5a (MD5) Previous issue date: 2014en
dc.identifier.doi10.1109/TPDS.2013.295en_US
dc.identifier.issn1045-9219en_US
dc.identifier.urihttp://hdl.handle.net/11693/26675en_US
dc.language.isoEnglishen_US
dc.publisherIEEE Computer Societyen_US
dc.relation.isversionofhttp://dx.doi.org/10.1109/TPDS.2013.295en_US
dc.source.titleIEEE Transactions on Parallel and Distributed Systemsen_US
dc.subjectData stream processingen_US
dc.subjectElasticityen_US
dc.subjectParallelizationen_US
dc.subjectData flow analysisen_US
dc.subjectData flow graphsen_US
dc.subjectProfitabilityen_US
dc.subjectApplication developersen_US
dc.subjectAuto-parallelizationen_US
dc.subjectData partitioningen_US
dc.subjectDistributed data stream processingen_US
dc.subjectParallel channelen_US
dc.subjectParallelizationsen_US
dc.subjectResource availabilityen_US
dc.subjectData communication systemsen_US
dc.titleElastic scaling for data stream processingen_US
dc.typeArticleen_US

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Elastic scaling for data stream processing.pdf
Size:
1.68 MB
Format:
Adobe Portable Document Format
Description:
Full printable version