Automatic performance evaluation of information retrieval systems using data fusion

Simple item page
buir.advisorGüvenir, Altay
dc.contributor.authorNuray, Rabia
dc.date.accessioned2016-07-01T10:58:36Z
dc.date.available2016-07-01T10:58:36Z
dc.date.issued2003
dc.descriptionCataloged from PDF version of article.en_US
dc.description.abstractThe empirical investigation of the effectiveness of information retrieval systems (search engines) requires a test collection composed of a set of documents, a set of query topics and a set of relevance judgments indicating which documents are relevant to which topics. The human relevance judgments are expensive and subjective. In addition to this databases and user interests change quickly. Hence there is a great need of automatic way of evaluating the performance of search engines. Furthermore, recent studies show that differences in human relevance assessments do not affect the relative performance of information retrieval systems. Based on these observations, in this thesis, we propose and use data fusion to replace human relevance judgments and introduce an automatic evaluation method and provide its comprehensive statistical assessment with several Text Retrieval Conference (TREC) systems which shows that the method results correlates positively and significantly with the actual human based evaluations. The major contributions of this thesis are: (1) an automatic information retrieval performance evaluation method that uses data fusion algorithms for the first time in the literature, (2) system selection methods for data fusion aiming even higher correlation among automatic and human-based results, (3) several practical implications stemming from the fact that the automatic precision values are strongly correlated with those of actual information retrieval systems.en_US
dc.description.provenanceMade available in DSpace on 2016-07-01T10:58:36Z (GMT). No. of bitstreams: 1 0002375.pdf: 4694840 bytes, checksum: 2d992945f1437b80b14e35d30febf71c (MD5) Previous issue date: 2003en
dc.description.statementofresponsibilityNuray, Rabiaen_US
dc.format.extentxvi, 95 leaves, graphics, tables, 30 cmen_US
dc.identifier.itemidBILKUTUPB071875
dc.identifier.urihttp://hdl.handle.net/11693/29371
dc.language.isoEnglishen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectautomatic performance evaluationen_US
dc.subjectTRECen_US
dc.subjectsystem performance predictionen_US
dc.subjectsocial welfare functionsen_US
dc.subjectinformation retrieval systemen_US
dc.subjectdata fusionen_US
dc.subject.lccZ699.A1 N87 2003en_US
dc.subject.lcshInformation storage and retrieval systems.en_US
dc.titleAutomatic performance evaluation of information retrieval systems using data fusionen_US
dc.typeThesisen_US
thesis.degree.disciplineComputer Engineering
thesis.degree.grantorBilkent University
thesis.degree.levelMaster's
thesis.degree.nameMS (Master of Science)

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
0002375.pdf
Size:
4.48 MB
Format:
Adobe Portable Document Format
Description:
Full printable version
Download

Collections

Graduate School of Engineering and Science