Nonlinear regression using second order methods

Abstract

We present a highly efficient algorithm for the online nonlinear regression problem. We process only the currently available data and do not reuse it, hence, there is no need for storage. For the nonlinear regression, we use piecewise linear modeling, where the regression space is partitioned into several regions and a linear model is fit to each region. As the first time in the literature, we use second order methods, e.g. Newton-Raphson Methods, and adaptively train both the region boundaries and the corresponding linear models. Therefore, we overcome the well known overfitting and underfitting problems. The proposed algorithm provides a substantial improvement in the performance compared to the state of the art.