Browsing by Subject "Document evaluation"

Filter results by typing the first few letters
Now showing 1 - 2 of 2
  • Thumbnail Image
    ItemOpen Access
    Ordinal covering using block designs
    (IEEE, 2010) Atmaca, Abdullah; Oruc, A.Y.
    A frequently encountered problem in peer review systems is to facilitate pairwise comparisons of a given set of documents by as few experts as possible. In [7], it was shown that, if each expert is assigned to review k documents then ⌈n(n-1)/k(k-1)⌉ experts are necessary and ⌈n(2n-k)/k 2⌉ experts are sufficient to cover all n(n-1)/2 pairs of n documents. In this paper, we show that, if √n ≤ k ≤ n/2 then the upper bound can be improved using a new assignnment method based on a particular family of balanced incomplete block designs. Specifically, the new method uses ⌈n(n+k)/k2⌉ experts where n/k is a prime power, n divides k2, and √n ≤ k ≤ n/2. When k = √n , this new method uses the minimum number of experts possible and for all other values of k, where √n < k ≤ n/2, the new upper bound is tighter than the general upper bound given in [7]. ©2010 IEEE.
  • Thumbnail Image
    ItemOpen Access
    Ordinal evaluation and assignment problems
    (IEEE, 2010) Atmaca, Abdullah; Oruç, A. Yavuz
    In many assignment problems, a set of documents such as research proposals, promotion dossiers, resumes of job applicants is assigned to a set of experts for ordinal evaluation, ranking, and classification. A desirable condition for such assignments is that every pair of documents is compared and ordered by one or more experts. This condition was modeled as an optimization problem and the number of pairs of documents was maximized for a given incidence relation between a set of documents and a set of experts using a set covering integer programming method in the literature[5]. In this paper, we use a combinatorial approach to derive lower bounds on the number of experts needed to compare all pairs of documents and describe assignments that asymptotically match these bounds. These results are not only theoretically interesting but also have practical implications in obtaining optimal assignments without using complex optimization techniques. ©2010 IEEE.