Iterative Hessian Sketch with momentum
Özaslan, İbrahim Kurban
Proceedings of the 44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019
Institute of Electrical and Electronics Engineers Inc.
7470 - 7474
Item Usage Stats
MetadataShow full item record
We propose a novel randomized linear least squares solver which is an improvement of Iterative Hessian Sketch and randomized preconditioning. In the proposed Momentum-IHS technique (M-IHS), Heavy Ball Method is used to accelerate the convergence of iterations. It is shown that for any full rank data matrix, rate of convergence depends on the ratio between the feature size and the sketch size. Unlike the Conjugate Gradient technique, the rate of convergence is unaffected by either the condition number or the eigenvalue spectrum of the data matrix. As demonstrated over many examples, the proposed M-IHS provides compatible performance with the state of the art randomized preconditioning methods such as LSRN or Blendenpik and yet, it provides a completely different perspective in the area of iterative solvers which can pave the way for future developments.
KeywordsIterative Hessian Sketch
First order iterative solvers