Show simple item record

dc.contributor.authorKahveci, Basrien_US
dc.contributor.authorGedik, Buğraen_US
dc.date.accessioned2021-02-25T06:56:49Z
dc.date.available2021-02-25T06:56:49Z
dc.date.issued2020
dc.identifier.issn0743-7315
dc.identifier.urihttp://hdl.handle.net/11693/75576
dc.description.abstractThis paper addresses the problem of auto-parallelization of streaming applications. We propose an online parallelization optimization algorithm that adjusts the degree of pipeline and data parallelism in a joint manner. We define an operator development API and a flexible parallel execution model to form a basis for the optimization algorithm. The operator interface unifies the development of different types of operators and makes operator properties visible in order to enable safe optimizations. The parallel execution model splits a data flow graph into regions. A region contains the longest sequence of compatible operators that are amenable to data parallelism as a whole and can be further parallelized with pipeline parallelism. We also develop a stream processing run-time, named Joker, to scale the execution of streaming applications in a safe, transparent, dynamic, and automatic manner. This ability is called organic adaptation. Joker implements the runtime machinery to execute a data flow graph with any parallelization configuration and most importantly change this configuration at run-time with low cost in the presence of partitioned stateful operators, in a way that is transparent to the application developers. Joker continuously monitors the run-time performance, and runs the optimization algorithm to resolve bottlenecks and scale the application by adjusting the degree of pipeline and data parallelism. The experimental evaluation based on micro-benchmarks and real-world applications showcase that our solution accomplishes elasticity by finding an effective parallelization configuration.en_US
dc.description.sponsorshipThis work was supported by the Scientific and Technical Re-search Council of Turkey (TUBITAK) under Grant No. 114E472.en_US
dc.language.isoEnglishen_US
dc.source.titleJournal of Parallel and Distributed Computingen_US
dc.relation.isversionofhttps://dx.doi.org/10.1016/j.jpdc.2019.10.012en_US
dc.subjectStream processingen_US
dc.subjectElasticityen_US
dc.subjectParallelizationen_US
dc.titleJoker: elastic stream processing with organic adaptationen_US
dc.typeArticleen_US
dc.departmentDepartment of Computer Engineeringen_US
dc.citation.spage205en_US
dc.citation.epage223en_US
dc.citation.volumeNumber137en_US
dc.identifier.doi10.1016/j.jpdc.2019.10.012en_US
dc.publisherElsevieren_US
dc.contributor.bilkentauthorKahveci, Basri
dc.contributor.bilkentauthorGedik, Buğra
dc.embargo.release2022-03-01


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record