Browsing by Subject "SDP"

Now showing 1 - 2 of 2

Open Access
Algorithms for sparsity constrained principal component analysis
(2023-07) Aktaş, Fatih Selim
The classical Principal Component Analysis problem consists of finding a linear transform that reduces the dimensionality of the original dataset while keeping most of the variation. Extra sparsity constraint sets most of the coefficients to zero which makes interpretation of the linear transform easier. We present two approaches to the sparsity constrained Principal Component Analysis. Firstly, we develop computationally cheap heuristics that can be deployed in very high-dimensional problems. Our heuristics are justified with linear algebra approximations and theoretical guarantees. Furthermore, we strengthen our algorithms by deploying the necessary conditions for the optimization model. Secondly, we use a non-convex log-sum penalty in the semidefinite space. We show a connection to the cardinality function and develop an algorithm, PCA Sparsified, to solve the problem locally via solving a sequence of convex optimization problems. We analyze the theoretical properties of this algorithm and comment on the numerical implementation. Moreover, we derive a pre-processing method that can be used with previous approaches. Finally, our findings from the numerical experiments we conducted show that our greedy algorithms scale to high dimensional problems easily while being highly competitive in many problems with state-of-art algorithms and even beating them uniformly in some cases. Additionally, we illustrate the effectiveness of PCA Sparsified on small dimensional problems in terms of variance explained. Although it is computationally very demanding, it consistently outperforms local and greedy approaches.
Open Access
PCA sparsified
(Society for Industrial and Applied Mathematics, 2023-08-09) Aktaş, Fatih S.; Pınar, Mustafa Çelebi
We propose an inverted approach to the Sparse Principal Component Analysis (SPCA) problem. Most previous research efforts focused on solving the problem of maximizing the variance subject to sparsity constraints or penalizing lack of sparsity. We focus on the problem of minimizing the number of nonzero elements of the loadings subject to a variance constraint. We derive a tractable approach for this problem using Semidefinite Programming (SDP). Our method minimizes a non-convex penalty function mimicking a cardinality penalty function more closely than the convex $ℓ_{1}$ norm which has been studied before. We develop a novel iterative weighted $ℓ_{1}$ norm minimization algorithm referred to as PCA Sparsified. We develop two algorithms to solve the weighted $ℓ_{1}$ norm minimization problem which have different efficiency estimates and computational complexity. Convergence properties of PCA Sparsified are studied. Connections to previously proposed methods are discussed. We introduce a preprocessing method to shrink the problem size which can also be used in previously proposed approaches. Numerical results based on careful implementation show the efficacy and potential of the proposed approach.