Losa, G.Kumar, V.Andrade, H.Gedik, BuğraHirzel, M.Soulé, R.Wu, K. -L.2016-02-082016-02-082012http://hdl.handle.net/11693/28172Date of Conference: 16 - 20 July 2012Conference name: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, DEBS'12Data stream processing applications are often expressed as data flow graphs, composed of operators connected via streams. This structured representation provides a simple yet powerful paradigm for building large-scale, distributed, high-performance applications. However, there are many tasks that require sharing data across operators, and across operators and the runtime using a less structured mechanism than point-to-point data flows. Examples include updating control variables, sending notifications, collecting metrics, building collective models, etc. In this paper we describe CAPSULE, which fills this gap. CAPSULE is a code generation and runtime framework that offers an easy to use and highly flexible framework for developers to realize shared variables (CAPSULE term for shared state) by specifying a data structure (at the programming-language level), and a few associated configuration parameters that qualify the expected usage scenario. Besides the easy of use and flexibility, CAPSULE offers the following important benefits: (1) Custom Code Generation - CAPSULE makes use of user-specified configuration parameters and information from the runtime to generate shared variable servers that are tailored for the specific usage scenario, (2) Composability - CAPSULE supports deployment time composition of the shared variable servers to achieve desired levels of scalability, performance and fault-tolerance, and (3) Extensibility - CAPSULE provides simple interfaces for extending the CAPSULE framework with more protocols, transports, caching mechanisms, etc. We describe the motivation for CAPSULE and its design, report on its implementation status, and then present experimental results. Copyright © 2012 ACM.EnglishConsistency modelsDistributed shared stateStream processingCaching mechanismCode GenerationComposabilityConfiguration parametersConsistency modelControl variableData flowData stream processingDeployment timeDistributed shared stateFlexible frameworkHigh performance applicationsRuntimesShared variablesStream processingStream processing systemsSystem supportsUsage scenariosData flow analysisData flow graphsData structuresFault toleranceNetwork componentsSoftware architectureDistributed parameter control systemsConference Paper10.1145/2335484.2335514