PCA sparsified

Abstract

We propose an inverted approach to the Sparse Principal Component Analysis (SPCA) problem. Most previous research efforts focused on solving the problem of maximizing the variance subject to sparsity constraints or penalizing lack of sparsity. We focus on the problem of minimizing the number of nonzero elements of the loadings subject to a variance constraint. We derive a tractable approach for this problem using Semidefinite Programming (SDP). Our method minimizes a non-convex penalty function mimicking a cardinality penalty function more closely than the convex $ℓ_{1}$ norm which has been studied before. We develop a novel iterative weighted $ℓ_{1}$ norm minimization algorithm referred to as PCA Sparsified. We develop two algorithms to solve the weighted $ℓ_{1}$ norm minimization problem which have different efficiency estimates and computational complexity. Convergence properties of PCA Sparsified are studied. Connections to previously proposed methods are discussed. We introduce a preprocessing method to shrink the problem size which can also be used in previously proposed approaches. Numerical results based on careful implementation show the efficacy and potential of the proposed approach.