Distributed k-Core view materialization and maintenance for large dynamic graphs
Author
Aksu, H.
Canim, M.
Chang, Yuan-Chi
Korpeoglu, I.
Ulusoy, O.
Date
2014-10Source Title
IEEE Transactions on Knowledge & Data Engineering
Print ISSN
1041-4347
Publisher
Institute of Electrical and Electronics Engineers
Volume
26
Issue
10
Pages
2439 - 2452
Language
English
Type
ArticleItem Usage Stats
109
views
views
113
downloads
downloads
Abstract
In graph theory, k-core is a key metric used to identify subgraphs of high cohesion, also known as the ‘dense’
regions of a graph. As the real world graphs such as social network graphs grow in size, the contents get richer and the
topologies change dynamically, we are challenged not only to materialize k-core subgraphs for one time but also to maintain
them in order to keep up with continuous updates. Adding to the challenge is that real world data sets are outgrowing the
capacity of a single server and its main memory. These challenges inspired us to propose a new set of distributed algorithms
for k-core view construction and maintenance on a horizontally scaling storage and computing platform. Our algorithms execute
against the partitioned graph data in parallel and take advantage of k-core properties to aggressively prune unnecessary
computation. Experimental evaluation results demonstrated orders of magnitude speedup and advantages of maintaining k-core
incrementally and in batch windows over complete reconstruction. Our algorithms thus enable practitioners to create and
maintain many k-core views on different topics in rich social network content simultaneously.
Keywords
K-coreGraph Theory
Distributed computing
Dynamic social networks
Random hypergraphs
Networks
Decomposition