Distributed block formation and layout for disk-based management of large-scale graphs
Distributed and Parallel Databases
23 - 53
Item Usage Stats
MetadataShow full item record
We are witnessing an enormous growth in social networks as well as in the volume of data generated by them. An important portion of this data is in the form of graphs. In recent years, several graph processing and management systems emerged to handle large-scale graphs. The primary goal of these systems is to run graph algorithms and queries in an efficient and scalable manner. Unlike relational data, graphs are semi-structured in nature. Thus, storing and accessing graph data using secondary storage requires new solutions that can provide locality of access for graph processing workloads. In this work, we propose a scalable block formation and layout technique for graphs, which aims at reducing the I/O cost of disk-based graph processing algorithms. To achieve this, we designed a scalable MapReduce-style method called ICBL, which can divide the graph into a series of disk blocks that contain sub-graphs with high locality. Furthermore, ICBL can order the resulting blocks on disk to further reduce non-local accesses. We experimentally evaluated ICBL to showcase its scalability, layout quality, as well as the effectiveness of automatic parameter tuning for ICBL. We deployed the graph layouts generated by ICBL on the Neo4j open source graph database, http://www.neo4j.org/ (2015) graph database management system. Our results show that the layout generated by ICBL reduces the query running times over Neo4j more than 2 × compared to the default layout. © 2017, Springer Science+Business Media New York.
KeywordsGraph management systems
Management information systems
Large scale graphs
Distributed database systems
Published Version (Please cite this version)http://dx.doi.org/10.1007/s10619-017-7191-3
Showing items related by title, author, creator and subject.
How content management problem of a remote laboratory system can be handled by integrating an open source learning management system? Problems and solutions Özdoğru, B.; Cagıltay, N. E. (2007)This paper represents the design and implementation of the integration process of an open source learning management system (LMS) to the remote laboratory platform. The reason of using a learning management system is to ...
Bayram V.; Tansel, B.T.; Yaman H. (Elsevier Ltd, 2015)Traffic management during an evacuation and the decision of where to locate the shelters are of critical importance to the performance of an evacuation plan. From the evacuation management authority's point of view, the ...
Ulusoy, Ö. (IEEE, 2007)Data management in Peer-to-Peer (P2P) systems is a complicated and challenging issue due to the scale of the network and highly transient population of peers. In this paper, we identify important research problems in P2P ...