Do Not Forget: Exploiting Stability-Plasticity Dilemma to Expedite Unsupervised SNN Training for Neuromorphic Processors

초록

This paper presents a novel early training termination technique that significantly improves the training speed and energy efficiency of unsupervised learning-based spiking neural networks (SNNs) by skipping redundant training samples. To achieve early termination, we leveraged the key observation that unsupervised SNNs tend to stably maintain previously learned information and systematically analyze the spike firing activity of the network during training. To make a training termination decision, we exploit the difference between the number of spikes generated by the previous and current input training samples. Our termination algorithm is adopted in an SNN using the spike-timing-dependent plasticity (STDP) learning rule for a pattern classification application. The proposed scheme made an early termination decision with insignificant accuracy performance loss by adequately ignoring redundant training samples. Specifically, it enhances the training speedup and energy efficiency by up to 5.07x and 5.14x, respectively, with less than 1 percent points (pp) accuracy loss compared to the baseline counterparts by skipping up to 80% of the training samples. Additionally, when employed in a VLSI-based neuromorphic chip environment, it exhibits up to 4.95x better energy efficiency than the baseline.

키워드