Efficient online training algorithms for recurrent neural networks

Simple item page
buir.advisorKozat, Süleyman Serdar
dc.contributor.authorVural, Nuri Mert
dc.date.accessioned2021-01-27T09:30:26Z
dc.date.available2021-01-27T09:30:26Z
dc.date.copyright2020-12
dc.date.issued2020-12
dc.date.submitted2021-01-26
dc.descriptionCataloged from PDF version of article.en_US
dc.descriptionThesis (Master's): Bilkent University, Department of Electrical and Electronics Engineering, İhsan Doğramacı Bilkent University, 2020.en_US
dc.descriptionIncludes bibliographical references (leaves 49-53).en_US
dc.description.abstractRecurrent Neural Networks (RNNs) are widely used for online regression due to their ability to learn nonlinear temporal dependencies. As an RNN model, Long-Short-Term-Memory Networks (LSTMs) are commonly preferred in prac-tice, since these networks are capable of learning long-term dependencies while avoiding the exploding gradient problem. On the other hand, the performance improvement of LSTMs usually comes with the price of their large parameter size, which makes their training significantly demanding in terms of computational and data requirements. In this thesis, we address the computational challenges of LSTM training. We introduce two training algorithms, designed for obtaining the online regression performance of LSTMs with less computational requirements than the state-of-the-art. The introduced algorithms are truly online, i.e., they do not assume any underlying data generating process and future information, except that the dataset is bounded. We discuss theoretical guarantees of the introduced algo-rithms, along with their asymptotic convergence behavior. Finally, we demon-strate their performance through extensive numerical studies on real and synthetic datasets, and show that they achieve the regression performance of LSTMs with significantly shorter training times.en_US
dc.description.provenanceSubmitted by Betül Özen (ozen@bilkent.edu.tr) on 2021-01-27T09:30:26Z No. of bitstreams: 1 10375991.pdf: 1751814 bytes, checksum: 52e338270467005d65194417f1c498a7 (MD5)en
dc.description.provenanceMade available in DSpace on 2021-01-27T09:30:26Z (GMT). No. of bitstreams: 1 10375991.pdf: 1751814 bytes, checksum: 52e338270467005d65194417f1c498a7 (MD5) Previous issue date: 2021-01en
dc.description.statementofresponsibilityby Nuri Mert Vuralen_US
dc.embargo.release2021-07-22
dc.format.extentxii, 74 leaves ; 30 cm.en_US
dc.identifier.itemidB150715
dc.identifier.urihttp://hdl.handle.net/11693/54922
dc.language.isoEnglishen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectLong-short-term-memoryen_US
dc.subjectRecurrent neural networksen_US
dc.subjectOnline opti-mizationen_US
dc.subjectKalman filteringen_US
dc.subjectSequential learningen_US
dc.titleEfficient online training algorithms for recurrent neural networksen_US
dc.title.alternativeYineleyici sinir ağları için verimli çevrimici eğitim algoritmalarıen_US
dc.typeThesisen_US
thesis.degree.disciplineElectrical and Electronic Engineering
thesis.degree.grantorBilkent University
thesis.degree.levelMaster's
thesis.degree.nameMS (Master of Science)

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
10375991.pdf
Size:
1.67 MB
Format:
Adobe Portable Document Format
Description:
Full printable version
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Graduate School of Engineering and Science