On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

Abstract

Long-term Cognitive Networks (LTCNs) are recurrent neural networks for modeling and simulation. Such networks can be trained in a synaptic or non-synaptic mode according to their goal. Non-synaptic learning refers to adjusting the transfer function parameters while preserving the weights connecting the neurons. In that regard, the Non-synaptic Backpropagation (NSBP) algorithm has proven successful in training LTCN-based models. Despite NSBP’s success, a question worthy of investigation is whether the backpropagation process is necessary when training these recurrent neural networks. This paper investigates this issue and presents three non-synaptic learning methods that modify the original algorithm. In addition, we perform a sensitivity analysis of both the NSBP’s hyperparameters and the LTCNs’ learnable parameters. The main conclusions of our study are i) the backward process attached to the NSBP algorithm is not necessary to train these recurrent neural systems, and ii) there is a non-synaptic learnable parameter that does not contribute significantly to the LTCNs’ performance.