Browsing by Author "Gökçesu, Kaan"

Now showing 1 - 6 of 6

Open Access
Asymptotically optimal contextual bandit algorithm using hierarchical structures
(Institute of Electrical and Electronics Engineers, 2018) Neyshabouri, Mohammadreza Mohaghegh; Gökçesu, Kaan; Gökçesu, Hakan; Özkan, Hüseyin; Kozat, Süleyman Serdar
We propose an online algorithm for sequential learning in the contextual multiarmed bandit setting. Our approach is to partition the context space and, then, optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data-driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithm based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design an efficient implementation for our algorithm using various hierarchical partitioning structures, such as lexicographical or arbitrary position splitting and binary trees (BTs) (and several other partitioning examples). For instance, in the case of BT partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (with respect to the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state of the art. Our experimental work extensively covers various scenarios ranging from bandit settings to multiclass classification with real and synthetic data. In these experiments, we show that our algorithm is highly superior to the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability. IEEE
Open Access
An efficient bandit algorithm for general weight assignments
(IEEE, 2017) Gökçesu, Kaan; Ergen, Tolga; Çiftçi, S.; Kozat, Süleyman Serdar
In this paper, we study the adversarial multi armed bandit problem and present a generally implementable efficient bandit arm selection structure. Since we do not have any statistical assumptions on the bandit arm losses, the results in the paper are guaranteed to hold in an individual sequence manner. The introduced framework is able to achieve the optimal regret bounds by employing general weight assignments on bandit arm selection sequences. Hence, this framework can be used for a wide range of applications.
Open Access
Estimating distributions varying in time in a universal manner
(IEEE, 2017) Gökçesu, Kaan; Manış, Eren; Kurt, Ali Emirhan; Yar, Ersin
We investigate the estimation of distributions with time-varying parameters. We introduce an algorithm that achieves the optimal negative likelihood performance against the true probability distribution. We achieve this optimum regret performance without any knowledge about the total change of the parameters of true distribution. Our results are guaranteed to hold in an individual sequence manner such that we have no assumptions on the underlying sequences. Apart from the regret bounds, through synthetic and real life experiments, we demonstrate substantial performance gains with respect to the state-of-the-art probability density estimation algorithms in the literature.
Open Access
Novelty detection using soft partitioning and hierarchical models
(IEEE, 2017) Ergen, Tolga; Gökçesu, Kaan; Şimşek, Mustafa; Kozat, Süleyman Serdar
In this paper, we study novelty detection problem and introduce an online algorithm. The algorithm sequentially receives an observation, generates a decision and then updates its parameters. In the first step, to model the underlying distribution, algorithm constructs a score function. In the second step, this score function is used to make the final decision for the observed data. After thresholding procedure is applied, the final decision is made. We obtain the score using versatile and adaptive nested decision tree. We employ nested soft decision trees to partition the observation space in an hierarchical manner. Based on the sequential performance, we optimize all the components of the tree structure in an adaptive manner. Although this in time adaptation provides powerful modeling abilities, it might suffer from overfitting. To circumvent overfitting problem, we employ the intermediate nodes of tree in order to generate subtrees and we then combine them in an adaptive manner. The experiments illustrate that the introduced algorithm significantly outperforms the state of the art methods.
Open Access
An online minimax optimal algorithm for adversarial multiarmed bandit problem
(Institute of Electrical and Electronics Engineers, 2018) Gökçesu, Kaan; Kozat, Süleyman Serdar
We investigate the adversarial multiarmed bandit problem and introduce an online algorithm that asymptotically achieves the performance of the best switching bandit arm selection strategy. Our algorithms are truly online such that we do not use the game length or the number of switches of the best arm selection strategy in their constructions. Our results are guaranteed to hold in an individual sequence manner, since we have no statistical assumptions on the bandit arm losses. Our regret bounds, i.e., our performance bounds with respect to the best bandit arm selection strategy, are minimax optimal up to logarithmic terms. We achieve the minimax optimal regret with computational complexity only log-linear in the game length. Thus, our algorithms can be efficiently used in applications involving big data. Through an extensive set of experiments involving synthetic and real data, we demonstrate significant performance gains achieved by the proposed algorithm with respect to the state-of-the-art switching bandit algorithms. We also introduce a general efficiently implementable bandit arm selection framework, which can be adapted to various applications.
Open Access
Sequential outlier detection based on incremental decision trees
(IEEE, 2019) Gökçesu, Kaan; Neyshabouri, Mohammadreza Mohaghegh; Gökçesu, Hakan; Serdar, Süleyman
We introduce an online outlier detection algorithm to detect outliers in a sequentially observed data stream. For this purpose, we use a two-stage filtering and hedging approach. In the first stage, we construct a multimodal probability density function to model the normal samples. In the second stage, given a new observation, we label it as an anomaly if the value of aforementioned density function is below a specified threshold at the newly observed point. In order to construct our multimodal density function, we use an incremental decision tree to construct a set of subspaces of the observation space. We train a single component density function of the exponential family using the observations, which fall inside each subspace represented on the tree. These single component density functions are then adaptively combined to produce our multimodal density function, which is shown to achieve the performance of the best convex combination of the density functions defined on the subspaces. As we observe more samples, our tree grows and produces more subspaces. As a result, our modeling power increases in time, while mitigating overfitting issues. In order to choose our threshold level to label the observations, we use an adaptive thresholding scheme. We show that our adaptive threshold level achieves the performance of the optimal prefixed threshold level, which knows the observation labels in hindsight. Our algorithm provides significant performance improvements over the state of the art in our wide set of experiments involving both synthetic as well as real data.