• About
  • Policies
  • What is open access
  • Library
  • Contact
Advanced search
      View Item 
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      JavaScript is disabled for your browser. Some features of this site may not work without it.

      Pipelined fission for stream programs with dynamic selectivity and partitioned state

      Thumbnail
      View / Download
      1023.8 Kb
      Author(s)
      Gedik, B.
      Özsema, H. G.
      Öztürk, Ö.
      Date
      2016
      Source Title
      Journal of Parallel and Distributed Computing
      Print ISSN
      0743-7315
      Publisher
      Academic Press
      Volume
      96
      Pages
      106 - 120
      Language
      English
      Type
      Article
      Item Usage Stats
      146
      views
      98
      downloads
      Abstract
      There is an ever increasing rate of digital information available in the form of online data streams. In many application domains, high throughput processing of such data is a critical requirement for keeping up with the soaring input rates. Data stream processing is a computational paradigm that aims at addressing this challenge by processing data streams in an on-the-fly manner, in contrast to the more traditional and less efficient store-and-then process approach. In this paper, we study the problem of automatically parallelizing data stream processing applications in order to improve throughput. The parallelization is automatic in the sense that stream programs are written sequentially by the application developers and are parallelized by the system. We adopt the asynchronous data flow model for our work, which is typical in Data Stream Processing Systems (DSPS), where operators often have dynamic selectivity and are stateful. We solve the problem of pipelined fission, in which the original sequential program is parallelized by taking advantage of both pipeline parallelism and data parallelism at the same time. Our pipelined fission solution supports partitioned stateful data parallelism with dynamic selectivity and is designed for shared-memory multi-core machines. We first develop a cost-based formulation that enables us to express pipelined fission as an optimization problem. The bruteforce solution of this problem takes a long time for moderately sized stream programs. Accordingly, we develop a heuristic algorithm that can quickly, but approximately, solve the pipelined fission problem. We provide an extensive evaluation studying the performance of our pipelined fission solution, including simulations as well as experiments with an industrial-strength DSPS. Our results show good scalability for applications that contain sufficient parallelism, as well as close to optimal performance for the heuristic pipelined fission algorithm.
      Keywords
      Auto-parallelization
      Data stream processing
      Fission
      Pipelining
      Application programs
      Data communication systems
      Data flow analysis
      Heuristic algorithms
      Optimization
      Problem solving
      Application developers
      Auto-parallelization
      Computational paradigm
      Data stream processing
      Fission
      Optimization problems
      Pipeline parallelisms
      Sequential programs
      Data handling
      Permalink
      http://hdl.handle.net/11693/36816
      Published Version (Please cite this version)
      http://dx.doi.org/10.1016/j.jpdc.2016.05.003
      Collections
      • Department of Computer Engineering 1435
      Show full item record

      Browse

      All of BUIRCommunities & CollectionsTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsThis CollectionTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartments

      My Account

      LoginRegister

      Statistics

      View Usage StatisticsView Google Analytics Statistics

      Bilkent University

      If you have trouble accessing this page and need to request an alternate format, contact the site administrator. Phone: (312) 290 1771
      © Bilkent University - Library IT

      Contact Us | Send Feedback | Off-Campus Access | Admin | Privacy