Browsing by Subject "Particle filtering"

Filter results by typing the first few letters
Now showing 1 - 2 of 2
  • Thumbnail Image
    ItemOpen Access
    Neural networks based online learning
    (IEEE, 2017) Ergen, Tolga; Kozat, Süleyman Serdar
    In this paper, we investigate online nonlinear regression and introduce novel algorithms based on the long short term memory (LSTM) networks. We first put the underlying architecture in a nonlinear state space form and introduce highly efficient particle filtering (PF) based updates, as well as, extended Kalman filter (EKF) based updates. Our PF based training method guarantees convergence to the optimal parameter estimation under certain assumptions. We achieve this performance with a computational complexity in the order of the first order gradient based methods by controlling the number of particles. The experimental results illustrate significant performance improvements achieved by the introduced algorithms with respect to the conventional methods.
  • Thumbnail Image
    ItemOpen Access
    Online training of LSTM networks in distributed systems for variable length data sequences
    (Institute of Electrical and Electronics Engineers, 2018) Ergen, T.; Kozat, Serdar
    In this brief, we investigate online training of long short term memory (LSTM) architectures in a distributed network of nodes, where each node employs an LSTM-based structure for online regression. In particular, each node sequentially receives a variable length data sequence with its label and can only exchange information with its neighbors to train the LSTM architecture. We first provide a generic LSTM-based regression structure for each node. In order to train this structure, we put the LSTM equations in a nonlinear state-space form for each node and then introduce a highly effective and efficient distributed particle filtering (DPF)-based training algorithm. We also introduce a distributed extended Kalman filtering-based training algorithm for comparison. Here, our DPF-based training algorithm guarantees convergence to the performance of the optimal LSTM coefficients in the mean square error sense under certain conditions. We achieve this performance with communication and computational complexity in the order of the first-order gradient-based methods. Through both simulated and real-life examples, we illustrate significant performance improvements with respect to the state-of-The-Art methods.