Discriminative fine-grained mixing for adaptive compression of data streams
Author(s)
Date
2014Source Title
IEEE Transactions on Computers
Print ISSN
0018-9340
Publisher
Institute of Electrical and Electronics Engineers
Volume
63
Issue
9
Pages
2228 - 2244
Language
English
Type
ArticleItem Usage Stats
207
views
views
197
downloads
downloads
Abstract
This paper introduces an adaptive compression algorithm for transfer of data streams across operators in stream processing systems. The algorithm is adaptive in the sense that it can adjust the amount of compression applied based on the bandwidth, CPU, and workload availability. It is discriminative in the sense that it can judiciously apply partial compression by selecting a subset of attributes that can provide good reduction in the used bandwidth at a low cost. The algorithm relies on the significant differences that exist among stream attributes with respect to their relative sizes, compression ratios, compression costs, and their amenability to application of custom compressors. As part of this study, we present a modeling of uniform and discriminative mixing, and provide various greedy algorithms and associated metrics to locate an effective setting when model parameters are available at run-time. Furthermore, we provide online and adaptive algorithms for real-world systems in which system parameters that can be measured at run-time are limited. We present a detailed experimental study that illustrates the superiority of discriminative mixing over uniform mixing. © 2013 IEEE.
Keywords
Adaptive compressionStream compression
Adaptive algorithms
Algorithms
Bandwidth
Bandwidth compression
Data communication systems
Data transfer
Mixing
Online systems
Parameter estimation
Real time systems
Greedy algorithms
Model parameters
Partial compressions
Real-world system
Relative sizes
Data compression
Permalink
http://hdl.handle.net/11693/25425Published Version (Please cite this version)
http://dx.doi.org/10.1109/TC.2013.103Collections
Related items
Showing items related by title, author, creator and subject.
-
A compression method based on compressive sampling for 3-D laser range scans of indoor environments
Dobrucalı, Oğuzcan; Barshan, Billur (Springer, Dordrecht, 2010)When 3-D models of environments need to be transmitted or stored, they should be compressed efficiently to increase the capacity of the communication channel or the storage medium. We propose a novel compression technique ... -
Content-based retrieval of historical Ottoman documents stored as textual images
Şaykol, E.; Sinop, A. K.; Güdükbay, Uğur; Ulusoy, Özgür; Çetin, A. Enis (IEEE, 2004)There is an accelerating demand to access the visual content of documents stored in historical and cultural archives. Availability of electronic imaging tools and effective image processing techniques makes it feasible to ... -
Novel compression algorithm based on sparse sampling of 3-D laser range scans
Dobrucali, O.; Barshan, B. (Oxford University Press, 2013)Three-dimensional models of environments can be very useful and are commonly employed in areas such as robotics, art and architecture, facility management, water management, environmental/industrial/urban planning and ...