Improved artificial neural network training with advanced methods
Author
Çatalbaş, Burak
Advisor
Morgül, Ömer
Date
2018-09Publisher
Bilkent University
Type
ThesisItem Usage Stats
137
views
views
125
downloads
downloads
Abstract
Artificial Neural Networks (ANNs) are used for different machine learning tasks
such as classification, clustering etc. They have been utilized in important tasks
and offering new services more and more in our daily lives. Learning capabilities
of these networks have accelerated significantly since 2000s, with the increasing
computational power and data amount. Therefore, research conducted on these
networks is renamed as Deep Learning, which emerged as a major research area
- not only in the neural networks, but also in the Machine Learning discipline.
For such an important research field, the techniques used in the training of these
networks can be seen as keys for more successful results. In this work, each part of
this training procedure is investigated by using of different and improved - sometimes
new - techniques on convolutional neural networks which classify grayscale
and colored image datasets. Advanced methods included the ones from the literature
such as He-truncated Gaussian initialization. In addition, our contributions
to the literature include ones such as SinAdaMax Optimizer, Dominantly Exponential
Linear Unit (DELU), He-truncated Laplacian initialization and Pyramid
Approach for Max-Pool layers. In the chapters of this thesis, success rates are
increased with the addition of these advanced methods accumulatively, especially
with DELU and SinAdaMax which are our contributions as upgraded methods.
In result, success rate thresholds for different datasets are met with simple
convolutional neural networks - which are improved with these advanced methods
and reached promising test success increases - within 15 to 21 hours (typically
less than a day). Thus, better performances are obtained by those different and
improved techniques are shown using well-known classification datasets.
Keywords
Arti cial Neural NetworksConvolutional Neural Networks
Deep Learning
Neural Network Training
CIFAR-10
MNIST