S³-TM : scalable streaming short text matching

Simple item page
buir.advisorFerhatosmano glu, Hakan
dc.contributor.authorBasık, Fuat
dc.date.accessioned2016-01-08T20:18:22Z
dc.date.available2016-01-08T20:18:22Z
dc.date.issued2014
dc.departmentDepartment of Computer Engineeringen_US
dc.descriptionAnkara : The Graduate School of Engineering and Science of Bilkent University, 2014.en_US
dc.descriptionThesis (Master's) -- Bilkent University, 2014.en_US
dc.descriptionIncludes bibliographical references leaves 46-49.en_US
dc.description.abstractMicro-blogging services have become major venues for information creation, as well as channels of information dissemination. Accordingly, monitoring them for relevant information is a critical capability. This is typically achieved by registering content-based subscriptions with the micro-blogging service. Such subscriptions are long running queries that are evaluated against the stream of posts. Given the popularity and scale of micro-blogging services like Twitter and Weibo, building a scalable infrastructure to evaluate these subscriptions is a challenge. To address this challenge, we present the S3-TM system for streaming short text matching. S3-TM is organized as a stream processing application, in the form of a data parallel ow graph designed to be run on a data center environment. It takes advantage of the structure of the publications (posts) and subscriptions to perform the matching in a scalable manner, without broadcasting publications or subscriptions to all of the matcher instances. The basic design of S3-TM uses a scoped multicast for publications and scoped anycast for subscriptions. To further improve throughput, we introduce publication routing algorithms that aim at minimizing the scope of the multicasts. The rst set of algorithms we develop are based on partitioning the word co-occurrence frequency graph, with the aim of routing posts that include commonly co-occurring words to a small set of matchers. While e ective, these algorithms fell short in balancing the load. To address this, we develop the SALB algorithm, which provides better load balance by modeling the load more accurately using the word-to-post bipartite graph. We also develop a subscription placement algorithm, called LASP, to group together similar subscriptions, in order to minimize the subscription matching cost. Furthermore, to achieve good scalability for increasing number of nodes, we introduce simple yet e ective techniques to handle workload skew. Finally, we introduce load shedding techniques for handling unexpected load spikes with small impact on the accuracy. Our experimental results show that S3-TM is scalable. Furthermore, the SALB algorithm provides more than 2:5 throughput compared to the baseline multicast and outperforms the graph partitioning based approaches.en_US
dc.description.degreeM.S.en_US
dc.description.statementofresponsibilityBasık, Fuaten_US
dc.embargo.release2017-01-16
dc.format.extentxii, 49 leaves, graphicsen_US
dc.identifier.itemidB149454
dc.identifier.urihttp://hdl.handle.net/11693/18335
dc.language.isoEnglishen_US
dc.publisherBilkent Universityen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectScalabilityen_US
dc.subjectStream Processingen_US
dc.subjectPublish/Subscribe Systemsen_US
dc.subjectText Matchingen_US
dc.subject.lccQA76.9.D338 B37 2014en_US
dc.subject.lcshData flow computing.en_US
dc.subject.lcshText processing (Computer science)en_US
dc.subject.lcshUbiquitous computing.en_US
dc.titleS³-TM : scalable streaming short text matchingen_US
dc.typeThesisen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
10061946.pdf
Size:
3.14 MB
Format:
Adobe Portable Document Format
Description:
Full printable version
Download

Collections

Dept. of Computer Engineering - Master's degree