Covolutional neural networks based on non-euclidean operators
buir.advisor | Çetin, Ahmet Enis | |
dc.contributor.author | Badawi, Diaa Hisham Jamil | |
dc.date.accessioned | 2018-01-10T13:32:26Z | |
dc.date.available | 2018-01-10T13:32:26Z | |
dc.date.copyright | 2018-01 | |
dc.date.issued | 2018-01 | |
dc.date.submitted | 2018-02-10 | |
dc.description | Cataloged from PDF version of article. | en_US |
dc.description | Thesis (M.S.): Bilkent University, Department of Electrical and Electronics Engineering, İhsan Doğramacı Bilkent University, 2018. | en_US |
dc.description | Includes bibliographical references (leaves 67-75). | en_US |
dc.description.abstract | Dot product-based operations in neural net feedforwarding passes are replaced with an ℓ₁-norm inducing operator, which itself is multiplication-free. The neural net, which is called AddNet, retains attributes of ℓ₁-norm based feature extraction schemes such as resilience against outliers. Furthermore, feedforwarding passes can be realized using fewer multiplication operations, which implies energy efficiency. The ℓ₁-norm inducing operator is differentiable w.r.t its operands almost everywhere. Therefore, it is possible to use it in neural nets that are to be trained through standard backpropagation algorithm. AddNet requires scaling (multiplicative) bias so that cost gradients do not explode during training. We present different choices for multiplicative bias: trainable, directly dependent upon the associated weights, or fixed. We also present a sparse variant of that operator, where partial or full binarization of weights is achievable. We ran our experiments over MNIST and CIFAR-10 datasets. AddNet could achieve results that are 0:1% less accurate than a ordinary CNN. Furthermore, trainable multiplicative bias helps the network to converge fast. In comparison with other binary-weights neural nets, AddNet achieves better results even with full or almost full weight magnitude pruning while keeping the sign information after training. As for experimenting on CIFAR-10, AddNet achieves accuracy 5% less than a ordinary CNN. Nevertheless, AddNet is more rigorous against impulsive noise data corruption and it outperforms the corresponding ordinary CNN in the presence of impulsive noise, even at small levels of noise. | en_US |
dc.description.provenance | Submitted by Betül Özen (ozen@bilkent.edu.tr) on 2018-01-10T13:32:26Z No. of bitstreams: 1 10175941_tez.pdf: 937743 bytes, checksum: 07701f568be375b30dc1221c72a68135 (MD5) | en |
dc.description.provenance | Made available in DSpace on 2018-01-10T13:32:26Z (GMT). No. of bitstreams: 1 10175941_tez.pdf: 937743 bytes, checksum: 07701f568be375b30dc1221c72a68135 (MD5) Previous issue date: 2018-01 | en |
dc.description.statementofresponsibility | by Diaa Hisham Jamil Badawi. | en_US |
dc.embargo.release | 2020-01-08 | |
dc.format.extent | xiv, 78 leaves : charts (some color) ; 30 cm | en_US |
dc.identifier.itemid | B157365 | |
dc.identifier.uri | http://hdl.handle.net/11693/35726 | |
dc.language.iso | English | en_US |
dc.rights | info:eu-repo/semantics/openAccess | en_US |
dc.subject | Deep Learning | en_US |
dc.subject | Convolutional Neural Network | en_US |
dc.subject | ℓ₁ Norm | en_US |
dc.subject | Energy Efficiency | en_US |
dc.subject | Binary Weights | en_US |
dc.subject | Impulsive Noise | en_US |
dc.title | Covolutional neural networks based on non-euclidean operators | en_US |
dc.title.alternative | Öklidce mensup olmayan operatörler bazında konvolüsyonel sinir ağıları | en_US |
dc.type | Thesis | en_US |
thesis.degree.discipline | Electrical and Electronic Engineering | |
thesis.degree.grantor | Bilkent University | |
thesis.degree.level | Master's | |
thesis.degree.name | MS (Master of Science) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- 10175941_tez.pdf
- Size:
- 915.76 KB
- Format:
- Adobe Portable Document Format
- Description:
- Full printable version
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 1.71 KB
- Format:
- Item-specific license agreed upon to submission
- Description: