• About
  • Policies
  • What is open access
  • Library
  • Contact
Advanced search
      View Item 
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      JavaScript is disabled for your browser. Some features of this site may not work without it.

      Joker: elastic stream processing with organic adaptation

      Thumbnail
      Embargo Lift Date: 2022-03-01
      View / Download
      1.3 Mb
      Author(s)
      Kahveci, Basri
      Gedik, Buğra
      Date
      2020
      Source Title
      Journal of Parallel and Distributed Computing
      Print ISSN
      0743-7315
      Publisher
      Elsevier
      Volume
      137
      Pages
      205 - 223
      Language
      English
      Type
      Article
      Item Usage Stats
      72
      views
      48
      downloads
      Abstract
      This paper addresses the problem of auto-parallelization of streaming applications. We propose an online parallelization optimization algorithm that adjusts the degree of pipeline and data parallelism in a joint manner. We define an operator development API and a flexible parallel execution model to form a basis for the optimization algorithm. The operator interface unifies the development of different types of operators and makes operator properties visible in order to enable safe optimizations. The parallel execution model splits a data flow graph into regions. A region contains the longest sequence of compatible operators that are amenable to data parallelism as a whole and can be further parallelized with pipeline parallelism. We also develop a stream processing run-time, named Joker, to scale the execution of streaming applications in a safe, transparent, dynamic, and automatic manner. This ability is called organic adaptation. Joker implements the runtime machinery to execute a data flow graph with any parallelization configuration and most importantly change this configuration at run-time with low cost in the presence of partitioned stateful operators, in a way that is transparent to the application developers. Joker continuously monitors the run-time performance, and runs the optimization algorithm to resolve bottlenecks and scale the application by adjusting the degree of pipeline and data parallelism. The experimental evaluation based on micro-benchmarks and real-world applications showcase that our solution accomplishes elasticity by finding an effective parallelization configuration.
      Keywords
      Stream processing
      Elasticity
      Parallelization
      Permalink
      http://hdl.handle.net/11693/75576
      Published Version (Please cite this version)
      https://dx.doi.org/10.1016/j.jpdc.2019.10.012
      Collections
      • Department of Computer Engineering 1510
      Show full item record

      Browse

      All of BUIRCommunities & CollectionsTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsCoursesThis CollectionTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsCourses

      My Account

      Login

      Statistics

      View Usage StatisticsView Google Analytics Statistics

      Bilkent University

      If you have trouble accessing this page and need to request an alternate format, contact the site administrator. Phone: (312) 290 2976
      © Bilkent University - Library IT

      Contact Us | Send Feedback | Off-Campus Access | Admin | Privacy