Nonstationary time series prediction with Markovian switching recurrent neural networks

Abstract

We investigate nonlinear prediction for nonstationary time series. In most real-life scenarios such as finance, retail, energy and economy applications, time se-ries data exhibits nonstationarity due to the temporally varying dynamics of the underlying system. This situation makes the time series prediction challenging in nonstationary environments. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in a Markovian way to model the nonstationary nature of the given data. Our model, Markovian RNN employs a hidden Markov model (HMM) for regime transitions, where each regime controls hidden state transitions of the recurrent cell inde-pendently. We jointly optimize the whole network in an end-to-end fashion. We demonstrate the significant performance gains compared to conventional methods such as Markov Switching ARIMA, RNN variants and recent statistical and deep learning-based methods through an extensive set of experiments with synthetic and real-life datasets. We also interpret the inferred parameters and regime belief values to analyze the underlying dynamics of the given sequences.