Browsing by Author "Vural, Nuri Mert"

Now showing 1 - 4 of 4

Open Access
Achieving online regression performance of LSTMs with simple RNNs
(Institute of Electrical and Electronics Engineers, 2021-06-17) Vural, Nuri Mert; İlhan, Fatih; Yılmaz, Selim Fırat; Ergüt, S.; Kozat, Süleyman Serdar
Recurrent neural networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, long short-term memory networks (LSTMs) are commonly preferred in practice, as these networks are capable of learning long-term dependencies while avoiding the vanishing gradient problem. However, due to their large number of parameters, training LSTMs requires considerably longer training time compared to simple RNNs (SRNNs). In this article, we achieve the online regression performance of LSTMs with SRNNs efficiently. To this end, we introduce a first-order training algorithm with a linear time complexity in the number of parameters. We show that when SRNNs are trained with our algorithm, they provide very similar regression performance with the LSTMs in two to three times shorter training time. We provide strong theoretical analysis to support our experimental results by providing regret bounds on the convergence rate of our algorithm. Through an extensive set of experiments, we verify our theoretical work and demonstrate significant performance improvements of our algorithm with respect to LSTMs and the other state-of-the-art learning models.
Open Access
Efficient online training algorithms for recurrent neural networks
(2020-12) Vural, Nuri Mert
Recurrent Neural Networks (RNNs) are widely used for online regression due to their ability to learn nonlinear temporal dependencies. As an RNN model, Long-Short-Term-Memory Networks (LSTMs) are commonly preferred in prac-tice, since these networks are capable of learning long-term dependencies while avoiding the exploding gradient problem. On the other hand, the performance improvement of LSTMs usually comes with the price of their large parameter size, which makes their training signiﬁcantly demanding in terms of computational and data requirements. In this thesis, we address the computational challenges of LSTM training. We introduce two training algorithms, designed for obtaining the online regression performance of LSTMs with less computational requirements than the state-of-the-art. The introduced algorithms are truly online, i.e., they do not assume any underlying data generating process and future information, except that the dataset is bounded. We discuss theoretical guarantees of the introduced algo-rithms, along with their asymptotic convergence behavior. Finally, we demon-strate their performance through extensive numerical studies on real and synthetic datasets, and show that they achieve the regression performance of LSTMs with signiﬁcantly shorter training times.
Open Access
Minimax optimal algorithms for adversarial bandit problem with multiple plays
(IEEE, 2019) Vural, Nuri Mert; Gökçesu, Hakan; Gökçesu, K.; Kozat, Süleyman Serdar
We investigate the adversarial bandit problem with multiple plays under semi-bandit feedback. We introduce a highly efficient algorithm that asymptotically achieves the performance of the best switching m-arm strategy with minimax optimal regret bounds. To construct our algorithm, we introduce a new expert advice algorithm for the multiple-play setting. By using our expert advice algorithm, we additionally improve the best-known high-probability bound for the multi-play setting by O(√(m)). Our results are guaranteed to hold in an individual sequence manner since we have no statistical assumption on the bandit arm gains. Through an extensive set of experiments involving synthetic and real data, we demonstrate significant performance gains achieved by the proposed algorithm with respect to the state-of-the-art algorithms.
Open Access
On feasibility of near-infrared spectroscopy for noninvasive blood glucose measurements
(SPIE, 2019-02) Vural, Nuri Mert; Yoleri, Y.; Torun, Hamdi
Non-invasive blood glucose measurement has long been desired since the invasive methods are not suitable to perform continuous monitoring. Near Infrared Spectroscopy is one of the most popular methods used in studies; however, despite more than 20 years of research, a practical and reliable noninvasive NIR glucose sensor is yet to be developed. In this study, we investigated the feasibility of NIRS towards the detection of glucose concentration. Although we can obtain adequate sensitivity, our measurements suffer from poor selectivity due to the fact that we can only detect the impurity level of water by NIRS due to strong water absorbance.