Fast and robust solution techniques for large scale linear least squares problems

Simple item page
buir.advisorArıkan, Orhan
dc.contributor.authorÖzaslan, İbrahim Kurban
dc.date.accessioned2020-08-26T07:31:08Z
dc.date.available2020-08-26T07:31:08Z
dc.date.copyright2020-07
dc.date.issued2020-07
dc.date.submitted2020-07-17
dc.descriptionCataloged from PDF version of article.en_US
dc.descriptionThesis (Master's): Bilkent University, Department of Electrical and Electronics Engineering, İhsan Doğramacı Bilkent University, 2020.en_US
dc.descriptionIncludes bibliographical references (leaves 99-107).en_US
dc.description.abstractMomentum Iterative Hessian Sketch (M-IHS) techniques, a group of solvers for large scale linear Least Squares (LS) problems, are proposed and analyzed in detail. Proposed M-IHS techniques are obtained by incorporating the Heavy Ball Acceleration into the Iterative Hessian Sketch algorithm and they provide significant improvements over the randomized preconditioning techniques. By using approximate solvers along with the iterations, the proposed techniques are capable of avoiding all matrix decompositions and inversions, which is one of the main advantages over the alternative solvers such as the Blendenpik and the LSRN. Similar to the Chebyshev Semi-iterations, the M-IHS variants do not use any inner products and eliminate the corresponding synchronization steps in hierarchical or distributed memory systems, yet the M-IHS converges faster than the Chebyshev Semi-iteration based solvers. Lower bounds on the required sketch size for various randomized distributions are established through the error analyses of the M-IHS variants. Unlike the previously proposed approaches to produce a solution approximation, the proposed M-IHS techniques can use sketch sizes that are proportional to the statistical dimension which is always smaller than the rank of the coefficient matrix. Additionally, hybrid schemes are introduced to estimate the unknown ℓ2-norm regularization parameter along with the iterations of the M-IHS techniques. Unlike conventional hybrid methods, the proposed Hybrid M-IHS techniques estimate the regularization parameter from the lower dimensional sub-problems that are constructed by random projections rather than the deterministic projections onto the Krylov Subspaces. Since the lower dimensional sub-problems that arise during the iterations of the Hybrid M-IHS variants are close approximations to the Newton sub-systems and the accuracy of their solutions increase exponentially, the parameters estimated from them rapidly converge to a proper regularization parameter for the full problem. In various numerical experiments conducted at several noise levels, the Hybrid M-IHS variants consistently estimated better regularization parameters and constructed solutions with less errors than the direct methods in far fewer iterations than the conventional hybrid methods. In large scale applications where the coefficient matrix is distributed over a memory array, the proposed Hybrid M-IHS variants provide improved efficiency by minimizing the number of distributed matrix-vector multiplications with the coefficient matrix.en_US
dc.description.provenanceSubmitted by Betül Özen (ozen@bilkent.edu.tr) on 2020-08-26T07:31:08Z No. of bitstreams: 1 10344618.pdf: 3182789 bytes, checksum: f8d48b97f3e86e6819de552dc68a86e8 (MD5)en
dc.description.provenanceMade available in DSpace on 2020-08-26T07:31:08Z (GMT). No. of bitstreams: 1 10344618.pdf: 3182789 bytes, checksum: f8d48b97f3e86e6819de552dc68a86e8 (MD5) Previous issue date: 2020-07en
dc.description.statementofresponsibilityby İbrahim Kurban Özaslanen_US
dc.format.extentxvii, 119 leaves : charts ; 30 cm.en_US
dc.identifier.itemidB160309
dc.identifier.urihttp://hdl.handle.net/11693/53936
dc.language.isoEnglishen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectLeast squaresen_US
dc.subjectTikhonov regularizationen_US
dc.subjectRidge regressionen_US
dc.subjectRandom projectionen_US
dc.subjectOblivious subspace embeddingsen_US
dc.subjectRandomized preconditioningen_US
dc.subjectAccelerationen_US
dc.subjectHybrid methodsen_US
dc.titleFast and robust solution techniques for large scale linear least squares problemsen_US
dc.title.alternativeBüyük ölçekli doğrusal en küçük kareler problemleri için hızlı ve gürbüz çözüm yöntemlerien_US
dc.typeThesisen_US
thesis.degree.disciplineElectrical and Electronic Engineering
thesis.degree.grantorBilkent University
thesis.degree.levelMaster's
thesis.degree.nameMS (Master of Science)

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
10344618.pdf
Size:
3.04 MB
Format:
Adobe Portable Document Format
Description:
Full printable version
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Graduate School of Engineering and Science