Active learning methods based on statistical leverage scores

Abstract

In many real-world machine learning applications, unlabeled data are abundant whereas the class labels are expensive and/or scarce. An active learner aims to obtain a model with high accuracy with as few labeled instances as possible by effectively selecting useful examples for labeling. We propose two novel active learning approaches for pool-based active learning setting: ALEVS for querying single example at each iteration and DBALEVS for querying a batch of examples. ALEVS and DBALEVS select the most in uential instance(s) based on statistical leverages scores of examples. The rank-k statistical leverage score of i-th row of an n x n kernel matrix K is the squared norm of the i-th row of the matrix U whose columns are the top-k eigenvectors of K. Statistical leverage scores are shown to be useful in matrix approximation algorithms in finding in uential rows of a matrix. ALEVS and DBALEVS assess the in uence of the examples by the statistical leverage scores of kernel matrix computed on the examples of the pool. Additionally, through maximizing a submodular set function at each iteration DBALEVS selects a diverse a set of examples that are highly in uential but are dissimilar to selected labeled set. Extensive experiments on diverse datasets show that the proposed methods, ALEVS and DBALEVS offer more effective strategies in comparison to other single and batch mode active learning approaches, respectively.