Browsing by Subject "Artificial neural network (ANN)"

Now showing 1 - 2 of 2

Open Access
Time and context sensitive optimization of machine learning models for sequential data prediction
(2024-07) Fazla, Arda
We investigate the nonlinear prediction of sequential time series data through the mixture/combination of machine learning models. First, we introduce a novel ensemble learning approach that effectively combines multiple base learners in a time-aware and context-sensitive manner. This process involves a weight optimization problem targeting a specific loss function while considering (non)convex constraints on the linear combination of base learners. These constraints are theoretically analyzed under known statistics and are automatically incorporated into the meta-learner as part of the optimization process during training. Next, we introduce a direct two-stage approach based on the combination of linear and nonlinear models, where we jointly optimize the parameters of both models to minimize the final regression error. By this joint optimization, we alleviate the well-known underfitting and overfitting problems in modeling sequential data. As the linear model, we use a traditional linear time series forecasting model (SARIMAX) and as the nonlinear model, we use boosted soft decision trees (Soft GBDT). For both of our approaches, we illustrate notable performance improvements on real-life data and well-known competition datasets compared to traditional ensemble/mixture techniques and state-of-the-art forecasting models in the machine learning literature. Additionally, we make the source code of both of our approaches publicly available to facilitate further research and comparison.
Open Access
Time-aware and context-sensitive ensemble learning for sequential data
(Institute of Electrical and Electronics Engineers, 2023-09-26) Fazla, Arda; Aydın, Mustafa E.; Kozat, Suleyman Serdar
We investigate sequential time series data through ensemble learning. Conventional ensemble algorithms and the recently introduced ones have provided significant performance improvements in widely publicized time series prediction competitions for stationary data. However, recent studies are inadequate in capturing the temporally varying statistics for non-stationary data. To this end, we introduce a novel approach using a meta learner that effectively combines base learners in both a time varying and context-dependent manner. Our approach is based on solving a weight optimization problem that minimizes a specific loss function with constraints on the linear combination of the base learners. The constraints are theoretically analyzed under known statistics and integrated into the learning procedure of the meta-learner as part of the optimization in an automated manner. We demonstrate significant performance improvements on real-life data and well-known competition datasets over the widely used conventional ensemble methods and the state-ofthe-art forecasting methods in the machine learning literature. Furthermore, we openly share the source code of our method to facilitate further research and comparison.