Varol, Emre2016-01-082016-01-082012http://hdl.handle.net/11693/15526Ankara : The Department of Computer Engineering and the Institute of Engineering and Science of Bilkent University, 2012.Thesis (Master's) -- Bilkent University, 2012.Includes bibliographical references leaves 37-39.In this thesis, we investigate containment detection, which is a generalized version of the well known near-duplicate detection problem concerning whether a document is a subset of another document. In text-based applications, there are three way of observing document containment: exact-duplicates, near-duplicates, or containments, where first two are the special cases of containment. To detect containments, we introduce CoDet, which is a novel algorithm that focuses particularly on containment problem. We also construct a test collection using a novel pooling technique, which enables us to make reliable judgments for the relative effectiveness of algorithms using limited human assessments. We compare its performance with four well-known near duplicate detection methods (DSC, full fingerprinting, I-Match, and SimHash) that are adapted to containment detection. Our algorithm is especially suitable for streaming news. It is also expandable to different domains. Experimental results show that CoDet mostly outperforms the other algorithms and produces remarkable results in detection of containments in text corpora.xii, 40 leaves, graphicsEnglishinfo:eu-repo/semantics/openAccessCorpus TreeNear-Duplicate DetectionSimilarityTest CollectionDocument ContainmentZ699 .V37 2012Information storage and retrieval systems.Information retrieval.Electronic data processing--Distributed processing.Similarity.ThesisB131817