Time-aware and context-sensitive ensemble learning for sequential data

Abstract

We investigate sequential time series data through ensemble learning. Conventional ensemble algorithms and the recently introduced ones have provided significant performance improvements in widely publicized time series prediction competitions for stationary data. However, recent studies are inadequate in capturing the temporally varying statistics for non-stationary data. To this end, we introduce a novel approach using a meta learner that effectively combines base learners in both a time varying and context-dependent manner. Our approach is based on solving a weight optimization problem that minimizes a specific loss function with constraints on the linear combination of the base learners. The constraints are theoretically analyzed under known statistics and integrated into the learning procedure of the meta-learner as part of the optimization in an automated manner. We demonstrate significant performance improvements on real-life data and well-known competition datasets over the widely used conventional ensemble methods and the state-ofthe-art forecasting methods in the machine learning literature. Furthermore, we openly share the source code of our method to facilitate further research and comparison.