Ordinal covering using block designs

Abstract

A frequently encountered problem in peer review systems is to facilitate pairwise comparisons of a given set of documents by as few experts as possible. In [7], it was shown that, if each expert is assigned to review k documents then ⌈n(n-1)/k(k-1)⌉ experts are necessary and ⌈n(2n-k)/k 2⌉ experts are sufficient to cover all n(n-1)/2 pairs of n documents. In this paper, we show that, if √n ≤ k ≤ n/2 then the upper bound can be improved using a new assignnment method based on a particular family of balanced incomplete block designs. Specifically, the new method uses ⌈n(n+k)/k2⌉ experts where n/k is a prime power, n divides k2, and √n ≤ k ≤ n/2. When k = √n , this new method uses the minimum number of experts possible and for all other values of k, where √n < k ≤ n/2, the new upper bound is tighter than the general upper bound given in [7]. ©2010 IEEE.