Evolutionary adaptation and dopamine modulated learning in Spiking Neural Networks

Abstract

Amidst the rise of Deep Learning (DL) in the last ten years, many decades with-standing problems such as image classification, machine translation and text generation received superhuman level solutions. Combining DL with other paradigms such as Reinforcement Learning (RL) led to even more astonishing achievements such as solving the game of Go, 3D Protein Folding and scene reconstruction. However, not only Artificial Neural Networks (ANNs) used in DL require a massive amount of data as the problems get complicated (curse of dimensionality) but also the learning algorithms used in ANNs such as backpropagation are biologically not plausible. On the other hand, Spiking Neural Networks (SNNs) are computationally powerful nonlinear dynamical systems that are biologically more credible. Thus, approaching problems using SNNs promises the possibility of transferring methods and discoveries from Neuroscience. In this work, we utilized a well known genetic algorithm called NeuroEvolution of Augmenting Topologies (NEAT) on SNNs to train agents which can solve various nonlinear problems such as XOR, pole balancing and food chasing. Afterwards, we ap-plied dopamine modulation on Spike-Timing Dependent Plasticity (STDP) and attested that dopamine modulated STDP can indeed solve harder problems such as discovering good and bad nutrition types while trying to catch and consume food. The gist of the problem is that the same food can be beneficial in one episode/trial and detrimental in another one. As a result, we have shown that NEAT applied to SNNs can solve various nonlinear tasks; however, it cannot solve problems where in-life adaptation and/or discovery is required.