Browsing by Subject "Online learning"

Now showing 1 - 20 of 65

Open Access
Achieving online regression performance of LSTMs with simple RNNs
(Institute of Electrical and Electronics Engineers, 2021-06-17) Vural, Nuri Mert; İlhan, Fatih; Yılmaz, Selim Fırat; Ergüt, S.; Kozat, Süleyman Serdar
Recurrent neural networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, long short-term memory networks (LSTMs) are commonly preferred in practice, as these networks are capable of learning long-term dependencies while avoiding the vanishing gradient problem. However, due to their large number of parameters, training LSTMs requires considerably longer training time compared to simple RNNs (SRNNs). In this article, we achieve the online regression performance of LSTMs with SRNNs efficiently. To this end, we introduce a first-order training algorithm with a linear time complexity in the number of parameters. We show that when SRNNs are trained with our algorithm, they provide very similar regression performance with the LSTMs in two to three times shorter training time. We provide strong theoretical analysis to support our experimental results by providing regret bounds on the convergence rate of our algorithm. Through an extensive set of experiments, we verify our theoretical work and demonstrate significant performance improvements of our algorithm with respect to LSTMs and the other state-of-the-art learning models.
Open Access
Actionable intelligence and online learning for semantic computing
(World Scientific Publishing Company, 2017) Tekin, Cem; van der Schaar, M.
As the world becomes more connected and instrumented, high dimensional, heterogeneous and time-varying data streams are collected and need to be analyzed on the fly to extract the actionable intelligence from the data streams and make timely decisions based on this knowledge. This requires that appropriate classifiers are invoked to process the incoming streams and find the relevant knowledge. Thus, a key challenge becomes choosing online, at run-time, which classifier should be deployed to make the best possible predictions on the incoming streams. In this paper, we survey a class of methods capable to perform online learning in stream-based semantic computing tasks: multi-armed bandits (MABs). Adopting MABs for stream mining poses, numerous new challenges requires many new innovations. Most importantly, the MABs will need to explicitly consider and track online the time-varying characteristics of the data streams and to learn fast what is the relevant information out of the vast, heterogeneous and possibly highly dimensional data streams. In this paper, we discuss contextual MAB methods, which use similarities in context (meta-data) information to make decisions, and discuss their advantages when applied to stream mining for semantic computing. These methods can be adapted to discover in real-time the relevant contexts guiding the stream mining decisions, and tract the best classifier in presence of concept drift. Moreover, we also discuss how stream mining of multiple data sources can be performed by deploying cooperative MAB solutions and ensemble learning. We conclude the paper by discussing the numerous other advantages of MABs that will benefit semantic computing applications.
Open Access
Active learning in context-driven stream mining with an application to ımage mining
(Institute of Electrical and Electronics Engineers, 2015-11) Tekin, C.; Schaar, Mihaela van der
We propose an image stream mining method in which images arrive with contexts (metadata) and need to be processed in real time by the image mining system (IMS), which needs to make predictions and derive actionable intelligence from these streams. After extracting the features of the image by preprocessing, IMS determines online the classifier to use on the extracted features to make a prediction using the context of the image. A key challenge associated with stream mining is that the prediction accuracy of the classifiers is unknown, since the image source is unknown; thus, these accuracies need to be learned online. Another key challenge of stream mining is that learning can only be done by observing the true label, but this is costly to obtain. To address these challenges, we model the image stream mining problem as an active, online contextual experts problem, where the context of the image is used to guide the classifier selection decision. We develop an active learning algorithm and show that it achieves regret sublinear in the number of images that have been observed so far. To further illustrate and assess the performance of our proposed methods, we apply them to diagnose breast cancer from the images of cellular samples obtained from the fine needle aspirate of breast mass. Our findings show that very high diagnosis accuracy can be achieved by actively obtaining only a small fraction of true labels through surgical biopsies. Other applications include video surveillance and video traffic monitoring.
Open Access
Adaptive ambulance redeployment via multi-armed bandits
(2019-09) Şahin, Ümitcan
Emergency Medical Services (EMS) provide the necessary resources when there is a need for immediate medical attention and play a signi cant role in saving lives in the case of a life-threatening event. Therefore, it is necessary to design an EMS system where the arrival times to calls are as short as possible. This task includes the ambulance redeployment problem that consists of the methods of deploying ambulances to certain locations in order to minimize the arrival time and increase the coverage of the demand points. As opposed to many conventional redeployment methods where the optimization is primary concern, we propose a learning-based approach in which ambulances are redeployed without any a priori knowledge on the call distributions and the travel times, and these uncertainties are learned on the way. We cast the ambulance redeployment problem as a multi-armed bandit (MAB) problem, and propose various context-free and contextual MAB algorithms that learn to optimize redeployment locations via exploration and exploitation. We investigate the concept of risk aversion in ambulance redeployment and propose a risk-averse MAB algorithm. We construct a data-driven simulator that consists of a graph-based redeployment network and Markov tra c model and compare the performances of the algorithms on this simulator. Furthermore, we also conduct more realistic simulations by modeling the city of Ankara, Turkey and running the algorithms in this new model. Our results show that given the same conditions the presented MAB algorithms perform favorably against a method based on dynamic redeployment and similarly to a static allocation method which knows the true dynamics of the simulation setup beforehand.
Open Access
Adaptive decision fusion based cooperative spectrum sensing for cognitive radio systems
(IEEE, 2011) Töreyin, B. U.; Yarkan, S.; Qaraqe, K. A.; Çetin, A. Enis
In this paper, an online Adaptive Decision Fusion (ADF) framework is proposed for the central spectrum awareness engine of a spectrum sensor network in Cognitive Radio (CR) systems. Online learning approaches are powerful tools for problems where drifts in concepts take place. Cooperative spectrum sensing in cognitive radio networks is such a problem where channel characteristics and utilization patterns change frequently. The importance of this problem stems from the requirement that secondary users must adjust their frequency utilization strategies in such a way that the communication performance of the primary users would not be degraded by any means. In the proposed framework, sensing values from several sensor nodes are fused together by weighted linear combination at the central spectrum awareness engine. The weights are updated on-line according to an active fusion method based on performing orthogonal projections onto convex sets describing power reading values from each sensor. The proposed adaptive fusion strategy for cooperative spectrum sensing can operate independent from the channel type between the primary user and secondary users. Results of simulations and experiments for the proposed method conducted in laboratory are also presented. © 2011 IEEE.
Open Access
Adaptive ensemble learning with confidence bounds
(Institute of Electrical and Electronics Engineers Inc., 2017) Tekin, C.; Yoon, J.; Schaar, M. V. D.
Extracting actionable intelligence from distributed, heterogeneous, correlated, and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on their locally collected data instances, and feed these predictions to an ensemble learner, which fuses them and issues a global prediction. However, most of these works do not provide performance guarantees or, when they do, these guarantees are asymptotic. None of these existing works provide confidence estimates about the issued predictions or rate of learning guarantees for the ensemble learner. In this paper, we provide a systematic ensemble learning method called Hedged Bandits, which comes with both long-run (asymptotic) and short-run (rate of learning) performance guarantees. Moreover, our approach yields performance guarantees with respect to the optimal local prediction strategy, and is also able to adapt its predictions in a data-driven manner. We illustrate the performance of Hedged Bandits in the context of medical informatics and show that it outperforms numerous online and offline ensemble learning methods.
Open Access
Adaptive ensemble learning with confidence bounds for personalized diagnosis
(AAAI Press, 2016) Tekin, Cem; Yoon, J.; Van Der Schaar, M.
With the advances in the field of medical informatics, automated clinical decision support systems are becoming the de facto standard in personalized diagnosis. In order to establish high accuracy and confidence in personalized diagnosis, massive amounts of distributed, heterogeneous, correlated and high-dimensional patient data from different sources such as wearable sensors, mobile applications, Electronic Health Record (EHR) databases etc. need to be processed. This requires learning both locally and globally due to privacy constraints and/or distributed nature of the multimodal medical data. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on their locally-collected data instances, and feed these predictions to an ensemble learner, which fuses them and issues a global prediction. However, most of these works do not provide performance guarantees or, when they do, these guarantees are asymptotic. None of these existing works provide confidence estimates about the issued predictions or rate of learning guarantees for the ensemble learner. In this paper, we provide a systematic ensemble learning method called Hedged Bandits, which comes with both long run (asymptotic) and short run (rate of learning) performance guarantees. Moreover, we show that our proposed method outperforms all existing ensemble learning techniques, even in the presence of concept drift.
Open Access
Asymptotically optimal contextual bandit algorithm using hierarchical structures
(Institute of Electrical and Electronics Engineers, 2018) Neyshabouri, Mohammadreza Mohaghegh; Gökçesu, Kaan; Gökçesu, Hakan; Özkan, Hüseyin; Kozat, Süleyman Serdar
We propose an online algorithm for sequential learning in the contextual multiarmed bandit setting. Our approach is to partition the context space and, then, optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data-driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithm based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design an efficient implementation for our algorithm using various hierarchical partitioning structures, such as lexicographical or arbitrary position splitting and binary trees (BTs) (and several other partitioning examples). For instance, in the case of BT partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (with respect to the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state of the art. Our experimental work extensively covers various scenarios ranging from bandit settings to multiclass classification with real and synthetic data. In these experiments, we show that our algorithm is highly superior to the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability. IEEE
Open Access
Combinatorial multi-armed bandit problem with probabilistically triggered arms: a case with bounded regret
(IEEE, 2017-11) Sarıtaç, A. Ömer; Tekin, Cem
In this paper, we study the combinatorial multi-armed bandit problem (CMAB) with probabilistically triggered arms (PTAs). Under the assumption that the arm triggering probabilities (ATPs) are positive for all arms, we prove that a simple greedy policy, named greedy CMAB (G-CMAB), achieves bounded regret. This improves the result in previous work, which shows that the regret is O (log T) under no such assumption on the ATPs. Then, we numerically show that G-CMAB achieves bounded regret in a real-world movie recommendation problem, where the action corresponds to recommending a set of movies, arms correspond to the edges between movies and users, and the goal is to maximize the total number of users that are attracted by at least one movie. In addition to this problem, our results directly apply to the online influence maximization (OIM) problem studied in numerous prior works.
Open Access
Computer network intrusion detection using various classifiers and ensemble learning
(IEEE, 2018) Mirza, Ali H.
In this paper, we execute anomaly detection over the computer networks using various machine learning algorithms. We then combine these algorithms to boost the overall performance. We implement three different types of classifiers, i.e, neural networks, decision trees and logistic regression. We then boost the overall performance of the intrusion detection algorithm using ensemble learning. In ensemble learning, we employ weighted majority voting scheme based on the individual classifier performance. We demonstrate a significant increase in the accuracy through a set of experiments KDD Cup 99 data set for computer network intrusion detection.
Open Access
Context-aware hierarchical online learning for performance maximization in mobile crowdsourcing
(Institute of Electrical and Electronics Engineers, 2018) Muller, S. K.; Tekin, Cem; Schaar, M.; Klein, A.
In mobile crowdsourcing (MCS), mobile users accomplish outsourced human intelligence tasks. MCS requires an appropriate task assignment strategy, since different workers may have different performance in terms of acceptance rate and quality. Task assignment is challenging, since a worker's performance 1) may fluctuate, depending on both the worker's current personal context and the task context and 2) is not known a priori, but has to be learned over time. Moreover, learning context-specific worker performance requires access to context information, which may not be available at a central entity due to communication overhead or privacy concerns. In addition, evaluating worker performance might require costly quality assessments. In this paper, we propose a context-aware hierarchical online learning algorithm addressing the problem of performance maximization in MCS. In our algorithm, a local controller (LC) in the mobile device of a worker regularly observes the worker's context, her/his decisions to accept or decline tasks and the quality in completing tasks. Based on these observations, the LC regularly estimates the worker's context-specific performance. The mobile crowdsourcing platform (MCSP) then selects workers based on performance estimates received from the LCs. This hierarchical approach enables the LCs to learn context-specific worker performance and it enables the MCSP to select suitable workers. In addition, our algorithm preserves worker context locally, and it keeps the number of required quality assessments low. We prove that our algorithm converges to the optimal task assignment strategy. Moreover, the algorithm outperforms simpler task assignment strategies in experiments based on synthetic and real data.
Open Access
Discovering influencers in opinion formation over social graphs
(Institute of Electrical and Electronics Engineers , 2023-03-23) Shumovskaia, V.; Kayaalp, M.; Cemri, Mert; Sayed, A. H.
The adaptive social learning paradigm helps model how networked agents are able to form opinions on a state of nature and track its drifts in a changing environment. In this framework, the agents repeatedly update their beliefs based on private observations and exchange the beliefs with their neighbors. In this work, it is shown how the sequence of publicly exchanged beliefs over time allows users to discover rich information about the underlying network topology and about the flow of information over the graph. In particular, it is shown that it is possible (i) to identify the influence of each individual agent to the objective of truth learning, (ii) to discover how well-informed each agent is, (iii) to quantify the pairwise influences between agents, and (iv) to learn the underlying network topology. The algorithm derived herein is also able to work under non-stationary environments where either the true state of nature or the graph topology are allowed to drift over time. We apply the proposed algorithm to different subnetworks of Twitter users, and identify the most influential and central agents by using their public tweets (posts).
Open Access
Distributed multi-agent online learning based on global feedback
(Institute of Electrical and Electronics Engineers, 2015-05-01) Tekin, C.; Zhang, S.; Schaar, Mihaela van der
Abstract—In this paper, we develop online learning algorithms that enable the agents to cooperatively learn how to maximize the overall reward in scenarios where only noisy global feedback is available without exchanging any information among themselves. We prove that our algorithms' learning regrets—the losses incurred by the algorithms due to uncertainty—are logarithmically increasing in time and thus the time average reward converges to the optimal average reward. Moreover, we also illustrate how the regret depends on the size of the action space, and we show that this relationship is influenced by the informativeness of the reward structure with regard to each agent's individual action. When the overall reward is fully informative, regret is shown to be linear in the total number of actions of all the agents. When the reward function is not informative, regret is linear in the number of joint actions. Our analytic and numerical results show that the proposed learning algorithms significantly outperform existing online learning solutions in terms of regret and learning speed. We illustrate how our theoretical framework can be used in practice by applying it to online Big Data mining using distributed classifiers.
Open Access
Distributed online learning via cooperative contextual bandits
(Institute of Electrical and Electronics Engineers, 2015-07-15) Tekin, C.; Schaar, Mihaela van der
In this paper, we propose a novel framework for decentralized, online learning by many learners. At each moment of time, an instance characterized by a certain context may arrive to each learner; based on the context, the learner can select one of its own actions (which gives a reward and provides information) or request assistance from another learner. In the latter case, the requester pays a cost and receives the reward but the provider learns the information. In our framework, learners are modeled as cooperative contextual bandits. Each learner seeks to maximize the expected reward from its arrivals, which involves trading off the reward received from its own actions, the information learned from its own actions, the reward received from the actions requested of others and the cost paid for these actions—taking into account what it has learned about the value of assistance from each other learner. We develop distributed online learning algorithms and provide analytic bounds to compare the efficiency of these with algorithms with the complete knowledge (oracle) benchmark (in which the expected reward of every action in every context is known by every learner). Our estimates show that regret—the loss incurred by the algorithm—is sublinear in time. Our theoretical framework can be used in many practical applications including Big Data mining, event detection in surveillance sensor networks and distributed online recommendation systems.
Open Access
An efficient and effective second-order training algorithm for LSTM-based adaptive learning
(IEEE, 2021-04-07) Vural, N. Mert; Ergüt, S.; Kozat, Süleyman S.
We study adaptive (or online) nonlinear regression with Long-Short-Term-Memory (LSTM) based networks, i.e., LSTM-based adaptive learning. In this context, we introduce an efficient Extended Kalman filter (EKF) based second-order training algorithm. Our algorithm is truly online, i.e., it does not assume any underlying data generating process and future information, except that the target sequence is bounded. Through an extensive set of experiments, we demonstrate significant performance gains achieved by our algorithm with respect to the state-of-the-art methods. Here, we mainly show that our algorithm consistently provides 10 to 45% improvement in the accuracy compared to the widely-used adaptive methods Adam, RMSprop, and DEKF, and comparable performance to EKF with a 10 to 15 times reduction in the run-time.
Open Access
Efficient online learning algorithms based on LSTM neural networks
(Institute of Electrical and Electronics Engineers, 2018) Ergen, Tolga; Kozat, Süleyman Serdar
We investigate online nonlinear regression and introduce novel regression structures based on the long short term memory (LSTM) networks. For the introduced structures, we also provide highly efficient and effective online training methods. To train these novel LSTM-based structures, we put the underlying architecture in a state space form and introduce highly efficient and effective particle filtering (PF)-based updates. We also provide stochastic gradient descent and extended Kalman filter-based updates. Our PF-based training method guarantees convergence to the optimal parameter estimation in the mean square error sense provided that we have a sufficient number of particles and satisfy certain technical conditions. More importantly, we achieve this performance with a computational complexity in the order of the first-order gradient-based methods by controlling the number of particles. Since our approach is generic, we also introduce a gated recurrent unit (GRU)-based approach by directly replacing the LSTM architecture with the GRU architecture, where we demonstrate the superiority of our LSTM-based approach in the sequential prediction task via different real life data sets. In addition, the experimental results illustrate significant performance improvements achieved by the introduced algorithms with respect to the conventional methods over several different benchmark real life data sets.
Open Access
Efficient online learning with improved LSTM neural networks
(Elsevier, 2020-04-14) Mirza, Ali H.; Kerpiçci, Mine; Kozat, Süleyman S.
We introduce efficient online learning algorithms based on the Long Short Term Memory (LSTM) networks that employ the covariance information. In particular, we introduce the covariance of the present and one-time step past input vectors into the gating structure of the LSTM networks. Additionally, we include the covariance of the output vector, and we learn their weight matrices to improve the learning performance of the LSTM networks where we also provide their updates. We reduce the number of system parameters through the weight matrix factorization where we convert the LSTM weight matrices into two smaller matrices in order to achieve high learning performance with low computational complexity. Moreover, we apply the introduced approach to the Gated Recurrent Unit (GRU) architecture. In our experiments, we illustrate significant performance improvements achieved by our methods on real-life datasets with respect to the vanilla LSTM and vanilla GRU networks.
Open Access
Emergency remote teaching in Turkey: a systematic literature review
(2022-12) İnal, Selin
The aim of this study is to systematically review the literature on Emergency Remote Teaching (ERT) during COVID-19 period in Turkey which started on March 23, 2020 and continued until the end of the spring term 2019-2020 in K12 and higher education context. The study sample consisted of 52 articles which were located from Scopus, ERIC and DergiPark databases through search criteria and examined under systematic literature review procedures. Articles are categorized according to their demographic data; methodology, data collection tools, size of the sample, sample type, level of the sample, curricular area, and digital platforms. Results indicated that qualitative research methods were the most preferred amongst the studies, conducted mostly in the higher education level. Sample sizes of the studies differed between 0-400 and small-scale research was the most popular. Amongst the articles, video conferencing tool Zoom was the most encountered digital tool. The research articles were reviewed to locate the changes happened within the teaching-learning cycle during ERT period regarding students and teachers of K12 and higher education contexts. Findings included following patterns; concepts of context as accessibility, flexibility of time and space, characteristics of home environments, internet and infrastructure problems, inequalities in possession of required technology; concepts of classroom processes as participation, use of materials and communication between students and instructors; concepts of input as content, students’ levels of interest and motivation, students’ study habits and students’ learning style. Implications for practice and implications for further research were given.
Open Access
End-to-end hybrid architectures for effective sequential data prediction
(2023-08) Aydın, Mustafa Enes
We investigate nonlinear prediction in an online setting and introduce two hybrid models that effectively mitigate, via end-to-end architectures, the need for hand-designed features and manual model selection issues of conventional nonlinear prediction/regression methods. Particularly, we first use an enhanced recurrent neural network (LSTM) to extract features from sequential signals, while pre-serving the state information, i.e., the history, and soft gradient boosted decision trees (sGBDT) to produce the final output. The connection is in an end-to-end fashion and we jointly optimize the whole architecture using stochastic gradient descent. Secondly, we again use recursive structures (LSTM) for automatic fea-ture extraction out of raw data but accompany it with a traditional linear time series model (SARIMAX) to deal with the intricacies of the sequential data, e.g., seasonality. The unification of the models is again in a joint manner; it is through a single state space and we optimize the entire architecture using particle filter-ing. The proposed frameworks are generic so that one can use other recurrent architectures, e.g., GRUs, and differentiable machine learning algorithms as well as time series models that have state space representations in lieu of the specific models presented. We demonstrate the learning behavior of the models on syn-thetic data and the significant performance improvements over the conventional methods and the disjoint counterparts over various real life datasets, with which we also show the generic nature of the frameworks. Furthermore, we openly share the source code of the proposed methods to facilitate further research.
Open Access
Energy consumption forecasting via order preserving pattern matching
(IEEE, 2014-12) Vanlı, N. Denizcan; Sayın, Muhammed O.; Yıldız, Hikmet; Göze, Tolga; Kozat, Süleyman S.
We study sequential prediction of energy consumption of actual users under a generic loss/utility function. Particularly, we try to determine whether the energy usage of the consumer will increase or decrease in the future, which can be subsequently used to optimize energy consumption. To this end, we use the energy consumption history of the users and define finite state (FS) predictors according to the relative ordering patterns of these past observations. In order to alleviate the overfitting problems, we generate equivalence classes by tying several states in a nested manner. Using the resulting equivalence classes, we obtain a doubly exponential number of different FS predictors, one among which achieves the smallest accumulated loss, hence is optimal for the prediction task. We then introduce an algorithm to achieve the performance of this FS predictor among all doubly exponential number of FS predictors with a significantly reduced computational complexity. Our approach is generic in the sense that different tying configurations and loss functions can be incorporated into our framework in a straightforward manner. We illustrate the merits of the proposed algorithm using the real life energy usage data. © 2014 IEEE.