A deterministic analysis of an online convex mixture of experts algorithm
IEEE Transactions on Neural Networks and Learning Systems
Institute of Electrical and Electronics Engineers Inc.
MetadataShow full item record
Please cite this item using this persistent URLhttp://hdl.handle.net/11693/21614
We analyze an online learning algorithm that adaptively combines outputs of two constituent algorithms (or the experts) running in parallel to estimate an unknown desired signal. This online learning algorithm is shown to achieve and in some cases outperform the mean-square error (MSE) performance of the best constituent algorithm in the steady state. However, the MSE analysis of this algorithm in the literature uses approximations and relies on statistical models on the underlying signals. Hence, such an analysis may not be useful or valid for signals generated by various real-life systems that show high degrees of nonstationarity, limit cycles and that are even chaotic in many cases. In this brief, we produce results in an individual sequence manner. In particular, we relate the time-accumulated squared estimation error of this online algorithm at any time over any interval to the one of the optimal convex mixture of the constituent algorithms directly tuned to the underlying signal in a deterministic sense without any statistical assumptions. In this sense, our analysis provides the transient, steady-state, and tracking behavior of this algorithm in a strong sense without any approximations in the derivations or statistical assumptions on the underlying signals such that our results are guaranteed to hold. We illustrate the introduced results through examples. © 2012 IEEE.
- Research Paper 7144
Showing items related by title, author, creator and subject.
Tse, S.S.H. (2008)We study the online bicriteria load balancing problem in this paper. We choose a system of distributed homogeneous file servers located in a cluster as the scenario and propose two online approximate algorithms for balancing ...
Akçay H.G.; Aksoy, S. (2008)The object-based analysis of remotely sensed imagery provides valuable spatial and structural information that is complementary to pixel-based spectral information in classification. In this paper, we present novel methods ...
SarıyÃ¼ce A.E.; Gedik B.; Jacques-Silva G.; Wu K.-L.; Ã atalyÃ¼rek Ã .V. (Springer New York LLC, 2016)A k-core of a graph is a maximal connected subgraph in which every vertex is connected to at least k vertices in the subgraph. k-core decomposition is often used in large-scale network analysis, such as community detection, ...