Browsing by Subject "Data flow analysis"

Now showing 1 - 5 of 5

Open Access
CAPSULE: Language and system support for efficient state sharing in distributed stream processing systems
(ACM, 2012) Losa, G.; Kumar, V.; Andrade, H.; Gedik, Buğra; Hirzel, M.; Soulé, R.; Wu, K. -L.
Data stream processing applications are often expressed as data flow graphs, composed of operators connected via streams. This structured representation provides a simple yet powerful paradigm for building large-scale, distributed, high-performance applications. However, there are many tasks that require sharing data across operators, and across operators and the runtime using a less structured mechanism than point-to-point data flows. Examples include updating control variables, sending notifications, collecting metrics, building collective models, etc. In this paper we describe CAPSULE, which fills this gap. CAPSULE is a code generation and runtime framework that offers an easy to use and highly flexible framework for developers to realize shared variables (CAPSULE term for shared state) by specifying a data structure (at the programming-language level), and a few associated configuration parameters that qualify the expected usage scenario. Besides the easy of use and flexibility, CAPSULE offers the following important benefits: (1) Custom Code Generation - CAPSULE makes use of user-specified configuration parameters and information from the runtime to generate shared variable servers that are tailored for the specific usage scenario, (2) Composability - CAPSULE supports deployment time composition of the shared variable servers to achieve desired levels of scalability, performance and fault-tolerance, and (3) Extensibility - CAPSULE provides simple interfaces for extending the CAPSULE framework with more protocols, transports, caching mechanisms, etc. We describe the motivation for CAPSULE and its design, report on its implementation status, and then present experimental results. Copyright © 2012 ACM.
Open Access
An efficient computation model for coarse grained reconfigurable architectures and its applications to a reconfigurable computer
(IEEE, 2010-07) Atak, Oğuzhan; Atalar, Abdullah
The mapping of high level applications onto the coarse grained reconfigurable architectures (CGRA) are usually performed manually by using graphical tools or when automatic compilation is used, some restrictions are imposed to the high level code. Since high level applications do not contain parallelism explicitly, mapping the application directly to CGRA is very difficult. In this paper, we present a middle level Language for Reconfigurable Computing (LRC). LRC is similar to assembly languages of microprocessors, with the difference that parallelism can be coded in LRC. LRC is an efficient language for describing control data flow graphs. Several applications such as FIR, multirate, multichannel filtering, FFT, 2D-IDCT, Viterbi decoding, UMTS and CCSDC turbo decoding, Wimax LDPC decoding are coded in LRC and mapped to the Bilkent Reconfigurable Computer with a performance (in terms of cycle count) close to that of ASIC implementations. The applicability of the computation model to a CGRA having low cost interconnection network has been validated by using placement and routing algorithms. © 2010 IEEE.
Open Access
Elastic scaling for data stream processing
(IEEE Computer Society, 2014) Gedik, B.; Schneider S.; Hirzel M.; Wu, Kun-Lung
This article addresses the profitability problem associated with auto-parallelization of general-purpose distributed data stream processing applications. Auto-parallelization involves locating regions in the application's data flow graph that can be replicated at run-time to apply data partitioning, in order to achieve scale. In order to make auto-parallelization effective in practice, the profitability question needs to be answered: How many parallel channels provide the best throughput? The answer to this question changes depending on the workload dynamics and resource availability at run-time. In this article, we propose an elastic auto-parallelization solution that can dynamically adjust the number of channels used to achieve high throughput without unnecessarily wasting resources. Most importantly, our solution can handle partitioned stateful operators via run-time state migration, which is fully transparent to the application developers. We provide an implementation and evaluation of the system on an industrial-strength data stream processing platform to validate our solution. © 1990-2012 IEEE.
Open Access
Pipelined fission for stream programs with dynamic selectivity and partitioned state
(Academic Press, 2016) Gedik, B.; Özsema, H. G.; Öztürk, Ö.
There is an ever increasing rate of digital information available in the form of online data streams. In many application domains, high throughput processing of such data is a critical requirement for keeping up with the soaring input rates. Data stream processing is a computational paradigm that aims at addressing this challenge by processing data streams in an on-the-fly manner, in contrast to the more traditional and less efficient store-and-then process approach. In this paper, we study the problem of automatically parallelizing data stream processing applications in order to improve throughput. The parallelization is automatic in the sense that stream programs are written sequentially by the application developers and are parallelized by the system. We adopt the asynchronous data flow model for our work, which is typical in Data Stream Processing Systems (DSPS), where operators often have dynamic selectivity and are stateful. We solve the problem of pipelined fission, in which the original sequential program is parallelized by taking advantage of both pipeline parallelism and data parallelism at the same time. Our pipelined fission solution supports partitioned stateful data parallelism with dynamic selectivity and is designed for shared-memory multi-core machines. We first develop a cost-based formulation that enables us to express pipelined fission as an optimization problem. The bruteforce solution of this problem takes a long time for moderately sized stream programs. Accordingly, we develop a heuristic algorithm that can quickly, but approximately, solve the pipelined fission problem. We provide an extensive evaluation studying the performance of our pipelined fission solution, including simulations as well as experiments with an industrial-strength DSPS. Our results show good scalability for applications that contain sufficient parallelism, as well as close to optimal performance for the heuristic pipelined fission algorithm.
Open Access
Sabit karasal mikrodalga LOS/NLOS radyo linklerin enterferans analizi
(IEEE, 2015-05) Göktaş, Polat; Topcu, Satılmış; Karaşan, Ezhan; Altıntaş, Ayhan
Bu çalışmada, NATO Band 3+ (1350-2690 MHz) ve NATO Band 4 (4400-5000 MHz) frekans bantlarında çalışan sabit karasal mikrodalga LOS (karasal görüş çizgisi)/ NLOS (ufuk ötesi) radyo linkleri için enterferans modellenmesi ele alınmıştır. Sabit karasal noktadan-noktaya haberleşme sistemlerinde enterferansa maruz kalan istasyonlardaki hem açık havadaki hem de yağmurdaki saçılmadan kaynaklanan enterferansın hesaplanması yapılmıştır. Ayrıca, ITU-R P.452 Tavsiyesinde bahsedilen açık havadaki enterferans kaybının hesabındaki enterferans yayılım mekanizmaları incelenmiştir. Enterferansa neden olan verici ve enterferansa maruz kalan alıcı istasyonların koordinat bilgileri, enterferansa neden olan verici ve enterferansa maruz kalan alıcı istasyonların antenlerinin yerden yükseklikleri, anten ayrımcılıgı, ˘ polarizasyon tipi, radyo kırılma indeksi, deniz seviyesinden ortalama kırıcılık, zaman yüzdesi, yığın kategorisi, sayısal arazi yükseklik haritası ve iklimsel veriler gibi enterferans yayılım parametreleri kullanılarak çeşitli mikrodalga radyo linkler için enterferans analizleri yapılmıştır.