Browsing by Subject "Least squares"

Now showing 1 - 8 of 8

Open Access
Büyük ölçekli doğrusal denklem sistemleri için hızlı ve gürbüz çözüm teknikleri
(IEEE, 2019-04) Özaslan, İbrahim K.; Pilancı, Mert; Arıkan, Orhan
Büyük ölçekli doğrusal sistemlerin veri matrisi, sütunlar arası yüksek ilintiye ve genellikle yüksek durum numaralarına sahiptir. Bilinmeyenlerin, ölçümlerden En Küçük Kareler (EKK) tekniğiyle üretilmesi, ölçüm gürültüsünün, sonucu kabul edilemez şekilde etkilemesine neden olmaktadır. Bu nedenle gürbüz çözüm tekniklerine ihtiyaç duyulmaktadır. Bu bildiride, yüksek durum numarasına sahip büyük ölçekli ölçüm matrislerinin yer aldığı doğrusal sistemlerin, Momentum - Yinelemeli Hessian Krokileme (Momentum - Iterative Hessian Sketch (M-IHS)) çözücüsü kullanılarak nasıl düzgelenebileceği incelenmiştir. Önerilen çözücü, tüm iterasyonlar için tek bir düzgeleme parametresi bulmak yerine, her bir iterasyon için düzgeleme parametresini başka bir parametre ayarı yapmadan otomatik olarak bulmakta ve daha sonra hızlı yaklaşım sağlayan momentum parametrelerini buna göre belirlemektedir. Yapılan analizde her ne kadar Genelleştirilmiş Çapraz Dogrulama (GCV) tekniği kullanılmış olsa da, M-IHS, bildiride açıklanan adımlar kullanılarak, herhangi bir risk tahmini ile düzgelenebilir.
Open Access
Cross-term free based bistatic radar system using sparse least squares
(SPIE, 2015) Sevimli, R. Akın; Çetin, A. Enis
Passive Bistatic Radar (PBR) systems use illuminators of opportunity, such as FM, TV, and DAB broadcasts. The most common illuminator of opportunity used in PBR systems is the FM radio stations. Single FM channel based PBR systems do not have high range resolution and may turn out to be noisy. In order to enhance the range resolution of the PBR systems algorithms using several FM channels at the same time are proposed. In standard methods, consecutive FM channels are translated to baseband as is and fed to the matched filter to compute the range-Doppler map. Multichannel FM based PBR systems have better range resolution than single channel systems. However superious sidelobe peaks occur as a side effect. In this article, we linearly predict the surveillance signal using the modulated and delayed reference signal components. We vary the modulation frequency and the delay to cover the entire range-Doppler plane. Whenever there is a target at a specific range value and Doppler value the prediction error is minimized. The cost function of the linear prediction equation has three components. The first term is the real-part of the ordinary least squares term, the second-Term is the imaginary part of the least squares and the third component is the l2-norm of the prediction coefficients. Separate minimization of real and imaginary parts reduces the side lobes and decrease the noise level of the range-Doppler map. The third term enforces the sparse solution on the least squares problem. We experimentally observed that this approach is better than both the standard least squares and other sparse least squares approaches in terms of side lobes. Extensive simulation examples will be presented in the final form of the paper.
Open Access
Fast and robust solution techniques for large scale linear least squares problems
(2020-07) Özaslan, İbrahim Kurban
Momentum Iterative Hessian Sketch (M-IHS) techniques, a group of solvers for large scale linear Least Squares (LS) problems, are proposed and analyzed in detail. Proposed M-IHS techniques are obtained by incorporating the Heavy Ball Acceleration into the Iterative Hessian Sketch algorithm and they provide signiﬁcant improvements over the randomized preconditioning techniques. By using approximate solvers along with the iterations, the proposed techniques are capable of avoiding all matrix decompositions and inversions, which is one of the main advantages over the alternative solvers such as the Blendenpik and the LSRN. Similar to the Chebyshev Semi-iterations, the M-IHS variants do not use any inner products and eliminate the corresponding synchronization steps in hierarchical or distributed memory systems, yet the M-IHS converges faster than the Chebyshev Semi-iteration based solvers. Lower bounds on the required sketch size for various randomized distributions are established through the error analyses of the M-IHS variants. Unlike the previously proposed approaches to produce a solution approximation, the proposed M-IHS techniques can use sketch sizes that are proportional to the statistical dimension which is always smaller than the rank of the coeﬃcient matrix. Additionally, hybrid schemes are introduced to estimate the unknown ℓ2-norm regularization parameter along with the iterations of the M-IHS techniques. Unlike conventional hybrid methods, the proposed Hybrid M-IHS techniques estimate the regularization parameter from the lower dimensional sub-problems that are constructed by random projections rather than the deterministic projections onto the Krylov Subspaces. Since the lower dimensional sub-problems that arise during the iterations of the Hybrid M-IHS variants are close approximations to the Newton sub-systems and the accuracy of their solutions increase exponentially, the parameters estimated from them rapidly converge to a proper regularization parameter for the full problem. In various numerical experiments conducted at several noise levels, the Hybrid M-IHS variants consistently estimated better regularization parameters and constructed solutions with less errors than the direct methods in far fewer iterations than the conventional hybrid methods. In large scale applications where the coeﬃcient matrix is distributed over a memory array, the proposed Hybrid M-IHS variants provide improved eﬃciency by minimizing the number of distributed matrix-vector multiplications with the coeﬃcient matrix.
Open Access
Goowe : geometrically optimum and online-weighted ensemble classifier for evolving data streams
(2016-07) Asl-Bonab, Hamed Rezanejad
Designing adaptive classifiers for an evolving data stream is a challenging task due to its size and dynamically changing nature. Combining individual classifiers in an online setting, the ensemble approach, is one of the well-known solutions. It is possible that a subset of classifiers in the ensemble outperforms others in a timevarying fashion. However, optimum weight assignment for component classifiers is a problem which is not yet fully addressed in online evolving environments. We propose a novel data stream ensemble classifier, called Geometrically Optimum and Online-Weighted Ensemble (GOOWE), which assigns optimum weights to the component classifiers using a sliding window containing the most recent data instances. We map vote scores of individual classifiers and true class labels into a spatial environment. Based on the Euclidean distance between vote scores and ideal-points, and using the linear least squares (LSQ) solution, we present a novel dynamic and online weighting approach. While LSQ is used for batch mode ensemble classifiers, it is the first time that we adapt and use it for online environments by providing a spatial modeling of online ensembles. In order to show the robustness of the proposed algorithm, we use real-world datasets and synthetic data generators using the MOA libraries. We compare our results with 8 state-ofthe- art ensemble classifiers in a comprehensive experimental environment. Our experiments show that GOOWE provides improved reactions to different types of concept drift compared to our baselines. The statistical tests indicate a significant improvement in accuracy, with conservative time and memory requirements.
Open Access
GOOWE: geometrically optimum and online-weighted ensemble classifier for evolving data streams
(Association for Computing Machinery, 2018-01-25) Bonab, H. R.; Can, Fazlı
Designing adaptive classifiers for an evolving data stream is a challenging task due to the data size and its dynamically changing nature. Combining individual classifiers in an online setting, the ensemble approach, is a well-known solution. It is possible that a subset of classifiers in the ensemble outperforms others in a time-varying fashion. However, optimum weight assignment for component classifiers is a problem which is not yet fully addressed in online evolving environments. We propose a novel data stream ensemble classifier, called Geometrically Optimum and Online-Weighted Ensemble (GOOWE), which assigns optimum weights to the component classifiers using a sliding window containing the most recent data instances. We map vote scores of individual classifiers and true class labels into a spatial environment. Based on the Euclidean distance between vote scores and ideal-points, and using the linear least squares (LSQ) solution, we present a novel, dynamic, and online weighting approach. While LSQ is used for batch mode ensemble classifiers, it is the first time that we adapt and use it for online environments by providing a spatial modeling of online ensembles. In order to show the robustness of the proposed algorithm, we use real-world datasets and synthetic data generators using the MOA libraries. First, we analyze the impact of our weighting system on prediction accuracy through two scenarios. Second, we compare GOOWE with 8 state-of-the-art ensemble classifiers in a comprehensive experimental environment. Our experiments show that GOOWE provides improved reactions to different types of concept drift compared to our baselines. The statistical tests indicate a significant improvement in accuracy, with conservative time and memory requirements.
Open Access
On robust solutions to linear least squares problems affected by data uncertainty and implementation errors with application to stochastic signal modeling
(Elsevier, 2004) Pınar, M. Ç.; Arıkan, Orhan
Engineering design problems, especially in signal and image processing, give rise to linear least squares problems arising from discretization of some inverse problem. The associated data are typically subject to error in these applications while the computed solution may only be implemented up to limited accuracy digits, i.e., quantized. In the present paper, we advocate the use of the robust counterpart approach of Ben-Tal and Nemirovski to address these issues simultaneously. Approximate robust counterpart problems are derived, which leads to semidefinite programming problems yielding stable solutions to overdetermined systems of linear equations affected by both data uncertainty and implementation errors, as evidenced by numerical examples from stochastic signal modeling.
Open Access
Robust least squares methods under bounded data uncertainties
(Academic Press, 2015) Vanli, N. D.; Donmez, M. A.; Kozat, S. S.
We study the problem of estimating an unknown deterministic signal that is observed through an unknown deterministic data matrix under additive noise. In particular, we present a minimax optimization framework to the least squares problems, where the estimator has imperfect data matrix and output vector information. We define the performance of an estimator relative to the performance of the optimal least squares (LS) estimator tuned to the underlying unknown data matrix and output vector, which is defined as the regret of the estimator. We then introduce an efficient robust LS estimation approach that minimizes this regret for the worst possible data matrix and output vector, where we refrain from any structural assumptions on the data. We demonstrate that minimizing this worst-case regret can be cast as a semi-definite programming (SDP) problem. We then consider the regularized and structured LS problems and present novel robust estimation methods by demonstrating that these problems can also be cast as SDP problems. We illustrate the merits of the proposed algorithms with respect to the well-known alternatives in the literature through our simulations.
Open Access
Structured least squares problems and robust estimators
(IEEE, 2010-10-22) Pilanci, M.; Arıkan, Orhan; Pinar, M. C.
A novel approach is proposed to provide robust and accurate estimates for linear regression problems when both the measurement vector and the coefficient matrix are structured and subject to errors or uncertainty. A new analytic formulation is developed in terms of the gradient flow of the residual norm to analyze and provide estimates to the regression. The presented analysis enables us to establish theoretical performance guarantees to compare with existing methods and also offers a criterion to choose the regularization parameter autonomously. Theoretical results and simulations in applications such as blind identification, multiple frequency estimation and deconvolution show that the proposed technique outperforms alternative methods in mean-squared error for a significant range of signal-to-noise ratio values.