Browsing by Subject "Long short term memory network"

Filter results by typing the first few letters
Now showing 1 - 3 of 3
  • Thumbnail Image
    ItemOpen Access
    A highly efficient recurrent neural network architecture for data regression
    (IEEE, 2018) Ergen, Tolga; Ceyani, Emir
    In this paper, we study online nonlinear data regression and propose a highly efficient long short term memory (LSTM) network based architecture. Here, we also introduce on-line training algorithms to learn the parameters of the introduced architecture. We first propose an LSTM based architecture for data regression. To diminish the complexity of this architecture, we use an energy efficient operator (ef-operator) instead of the multiplication operation. We then factorize the matrices of the LSTM network to reduce the total number of parameters to be learned. In order to train the parameters of this structure, we introduce online learning methods based on the exponentiated gradient (EG) and stochastic gradient descent (SGD) algorithms. Experimental results demonstrate considerable performance and efficiency improvements provided by the introduced architecture.
  • Thumbnail Image
    ItemOpen Access
    Neural networks based online learning
    (IEEE, 2017) Ergen, Tolga; Kozat, Süleyman Serdar
    In this paper, we investigate online nonlinear regression and introduce novel algorithms based on the long short term memory (LSTM) networks. We first put the underlying architecture in a nonlinear state space form and introduce highly efficient particle filtering (PF) based updates, as well as, extended Kalman filter (EKF) based updates. Our PF based training method guarantees convergence to the optimal parameter estimation under certain assumptions. We achieve this performance with a computational complexity in the order of the first order gradient based methods by controlling the number of particles. The experimental results illustrate significant performance improvements achieved by the introduced algorithms with respect to the conventional methods.
  • Thumbnail Image
    ItemOpen Access
    Online distributed nonlinear regression via neural networks
    (IEEE, 2017) Ergen, Tolga; Kozat, Süleyman Serdar
    In this paper, we study the nonlinear regression problem in a network of nodes and introduce long short term memory (LSTM) based algorithms. In order to learn the parameters of the LSTM architecture in an online manner, we put the LSTM equations into a nonlinear state space form and then introduce our distributed particle filtering (DPF) based training algorithm. Our training algorithm asymptotically achieves the optimal training performance. In our simulations, we illustrate the performance improvement achieved by the introduced algorithm with respect to the conventional methods.