A unified approach to universal prediction: Generalized upper and lower bounds
Author
Vanli, N. D.
Kozat, S. S.
Date
2015Source Title
IEEE Transactions on Neural Networks and Learning Systems
Print ISSN
0216-2237X
Publisher
Institute of Electrical and Electronics Engineers Inc.
Volume
26
Issue
3
Pages
646 - 651
Language
English
Type
ArticleItem Usage Stats
129
views
views
78
downloads
downloads
Abstract
We study sequential prediction of real-valued, arbitrary, and unknown sequences under the squared error loss as well as the best parametric predictor out of a large, continuous class of predictors. Inspired by recent results from computational learning theory, we refrain from any statistical assumptions and define the performance with respect to the class of general parametric predictors. In particular, we present generic lower and upper bounds on this relative performance by transforming the prediction task into a parameter learning problem. We first introduce the lower bounds on this relative performance in the mixture of experts framework, where we show that for any sequential algorithm, there always exists a sequence for which the performance of the sequential algorithm is lower bounded by zero. We then introduce a sequential learning algorithm to predict such arbitrary and unknown sequences, and calculate upper bounds on its total squared prediction error for every bounded sequence. We further show that in some scenarios, we achieve matching lower and upper bounds, demonstrating that our algorithms are optimal in a strong minimax sense such that their performances cannot be improved further. As an interesting result, we also prove that for the worst case scenario, the performance of randomized output algorithms can be achieved by sequential algorithms so that randomized output algorithms do not improve the performance. © 2012 IEEE.
Keywords
Online learningComputation theory
Forecasting
Sequential switching
Computational learning theory
Lower and upper bounds
Online learning
Sequential learning algorithm
Sequential prediction
Squared prediction errors
Upper and lower bounds
Worst-case performance
Algorithms