Hebbian Learning in FPGA Silicon Neuronal Network

This paper describes a digital silicon neuronal network trained by the Hebbian learning rule that can execute the auto-associative memory. In our previous work, a fully connected network of 256 silicon neurons based on the digital spiking silicon neuron (DSSN) model and kinetic-model-based silicon synapses were implemented. In this work, we added circuit modules that append Hebbian learning function and fitted it to a Xilinx Virtex 6 XC6VSX315T FPGA device. The performances of auto-associative memory with several spike-time-dependent Hebbian learning rules and the correlation rule are compared. The results show that Hebbian learning rules that model both synaptic potentiation and depression improve the retrieval probability in our silicon neuronal network.


Introduction
Sensory experiences are thought to be configuring the nerve system.The synaptic plasticity underlies this phenomenon, which refers to the change of the connection strength between two neurons.From the view point of molecular biology, the modification of the neurotransmitter release and the receptors can cause the plasticity of the synapses (1,2) .The modification that lasts for long time has two types: long-term potentiation (LTP) and long-term depression (LTD) which respectively leads to reinforced and weakened synapses.Physiological experiments revealed LTP and LTD in the hippocampus and the cortex (3,4) .The Hebbian learning rule firstly described that the connection from one cell to another is facilitated if their activities are correlated (5) .Several artificial neural networks with the Hebbian rule are simulated on PCs or designed by electronic circuits (7,8) .However, these kinds of systems mimic restricted aspects of the nervous system because they simplify the timing information of spikes by using discrete time models such as the two-state neuron model.Continuous-time models such as integrate-and-fire (I&F) model are chosen to build neuronal networks in order to overcome the above issue (9,10) .
We proposed a silicon neuronal network implemented in an FPGA device.It is composed of silicon neurons and synapses that are optimized for implementation by digital arithmetical circuits and capable of real-time operation in entry-level FGPA devices.The model of these silicon neurons was designed in the viewpoint of nonlinear dynamics and can reproduce the graded spike response in the Class II neurons in the Hodgkin's classification (11) .The silicon synapse model describes the transmitter release, receptor activation and the generation of synaptic current.In this work, we train our silicon neuronal network by the Hebbian learning rule and estimate the performance of associative memory.
Section 2 explains the model of the Hebbian learning rule used in this paper.Section 3 presents the implementation of the circuit of our silicon neuronal network with the Hebbian learning ability.Section 4 describes the associative memory we performed and compares their performances.Conclusion and the future plan of our work are presented in section 5.

Hebbian learning rules
The Hebbian learning rule, proposed in 1949, supposes that if input from neuron A causes the firing of neuron B, then the synapse from A to B tends to be strengthened (5) .Exponential learning windows are a natural choice for DOI: 10.12792/icisip2013.020describing the Hebbian rule for spike-timing dependent learning (6) .A most simple formulation is, where ∆W is the change of synaptic weight W and ∆t is the time difference between the pre-and post-synaptic spikes.Parameters A + and τ + represent the amplitude and the decay constant.
The above basic Hebbian rule only considers the synaptic potentiation.The synaptic depression, which is also an important process of learning, can be modeled by exponential weight decay (6) .A simple formulation of a model with both of the potentiation and the depression is as follows.
where A − and τ − are the amplitude and the decay constant for the synaptic depression.We also use more simplified model as in Eq. ( 3) because simpler models require less hardware resources.
Figure 1 shows the learning windows of Eqs. ( 1), (2), and . Learning rules in Eqs. ( 2) and (3) (we refer to them as anti-combined (AC) and decay-combined (DC) rules, respectively) can lead to decreasing synaptic change while the basic one cannot.When the time difference becomes large, the decreasing rate is constant in the DC rule while it decays in the AC rule.

FPGA silicon neuronal network 3.1 Model of silicon neuronal network
The model of our silicon neuronal network is composed of the Digital Spiking Silicon Neuron (DSSN) model (12) and a silicon synapse model proposed in our previous work (13) .The DSSN model is two-variable differential equations optimized for digital arithmetic circuit implementation.Its dynamics can be configured by selecting appropriate parameter values.Both Class I and II neurons in the Hodgkin's classification (14) can be reproduced by the two sets of parameter values listed in appendix.The equations of this model are, where  and  denote the membrane potential and a slow variable that represents the activity of ionic channels, respectively.Parameter  0 is a bias constant and   is the weighed sum of postsynaptic input from silicon synapse.Parameters  and  are time constants.Parameters r,   ,  ,  ,   ,   ,   for x=n and p, are the parameters that are responsible to the dynamical behaviors.
The synaptic model is a kinetic-model-based one that describes the transmitter release and the postsynaptic current generation in response to the presynaptic membrane potential.Its equation is, where,   and [] are the postsynaptic current and the amount of the released transmitter per impinging spike, respectively.Parameters  and  are the forward and backward rate constants which represent the rate of the receptors transitioning from the closed state to the open state and its opposite, respectively.The DSSN model in the Class II mode responds to stimulus input in a graded manner: the peak of  in a spike varies depending on the strength of stimulus current.The amount of the released transmitter is controlled by the stimulus current due to this graded response property.The stimulus input to the i-th silicon neuron    is calculated by adding the postsynaptic currents as follows, wehre  , is the connection weight from the j-th to the i-th silicon neurons.And c is a coefficient used to restrict    to an appropriate range and ensure that neurons fire regularly.The values of parameters for Class I and II neurons are listed in the appendix.

FPGA implementation
We designed a fully connected network which has the pipelined and parallel structure as shown in Fig. 2. It contains multiple (N f ) silicon neuronal network modules (SNNMs).Each SNNM calculates the system variables for N v silicon neurons in sequence.The SNNM is composed of four units: a DSSN, a silicon synapse, an accumulator and a learning units.The DSSN and the silicon synapse units update the states of the silicon neurons and the silicon synapses in response to stimulus input    from accumulator unit.The learning units modify the connection weights  , according to the Hebbian learning rule when it detects spikes.

Auto-associative memory 4.1 Configuration of the network
The auto-associative memory retrieves a stored pattern with the help of its fragments.It was originally investigated in a fully connected neuronal networks with the correlation learning (15) .Then some online learning rules such as Hebbian learning rules were introduced (16,17,18) .We executed the auto-associative memory by our silicon neuronal network with the correlation learning in the previous work (13) .In this paper, we report the performance in this task of our silicon neuronal network equipped with the Hebbian learning ability.We put our network firstly in the learning stage and then in the retrieval stage.In the learning stage, four stored patterns shown in Fig. 3(A) and their reversed patterns were presented to our network alternately in the following order: Here, the reversed patterns (x ̅ corresponds to x) are used to increase the contrast between two pixels with different colors to ensure an efficient learning by the Hebbian rule.The coupled patterns x and x ̅ (for x = 1,2,3,4) repeat 8 times and interval between two different coupled patterns is set to 24ms.This period was selected so that stimuli avoid Fig. 4. Transition of connection weights with L1-5 rules.the refractory period of silicon neurons induced by the last stimulus.One pixel in patterns corresponds to one silicon neuron in our network.A pulse current is added to the neuron if its corresponding pixel is black.The amplitude and the high and the rest periods of this pulse current are 0.125, 6ms and 18ms, respectively.In this paper, each silicon neuron is in the Class I mode.We designed 5 types of Hebbian learning rules L1 to L5 based on Eqs. ( 1)-( 3) with soft or hard bounds that limit the weights in a finite range.Their update equations are, Here,   () = The hard bound in L5 is that the update of weights is stopped when one of the weights reaches a limit,   or   which equal to -1 and 1 respectively.The weights are updated from their initial value 0 according to the above 5 Hebbian learning rules and the update process stops when weights are unchanged for 600ms.The values of parameters in Eqs. ( 10)-( 14) are listed in the appendix.

Simulation results
We simulated our silicon neuronal network with C++ programs to estimate the performance.Figure 4 shows the transition of connection weights to the silicon neuron 1 (W 1,1 , … , W 1,256 ) when our network is trained by the above 5 Hebbian learning rules.With the basic Hebbian learning rule with soft bounds (L1), the weights increased and reached their maximum value, 1(Fig.4(A)).The weights also increased based on the DC Hebbian learning rule with soft bounds (L2) in Fig. 4(B).However, the decay factor  − and its relevant soft bound   slowed down their growth speed and decreased their maximum values.By the AC Hebbian learning rule with two soft bounds Fig. 5. Averaged difference of weights between the Hebbian learning rules (L1-5) and the correlation learning rule.
corresponding to LTP and LTD (L3), the weights stayed within the range of [-0.4,0.2] (Fig. 4(C)).In another AC Hebbian learning rule (L4), two soft bounds commonly effect on LTP and LTD together.This updated some of the weights to their boundary values (Fig. 4(D)).The AC Hebbian learning rule with hard bound (L5) modified the weights more quickly than the soft bounds rules (L1-4) (Fig. 4(E)).
We have verified that our silicon neuronal network with the correlation learning rule can successfully execute auto-associative memory in our previous work (13) .Here, the connection weights were calculated by the correlation learning rule according to the following equations: where p is the number of stored patterns, which is 4 in this work, and    represents the i-th pixel in the u-th stored pattern, which is 1 or -1 when it is black or white, respectively.Figure 5 plots the averaged difference of weights between the Hebbian learning rules (L1-5) and the correlation learning rule ( dW ).For L1 and L2 rules, dW grew monotonically as the learning proceeds.It indicates the weights produced by L1 and L2 are completely different from those by the correlation rule (Fig. 5(A, B)).For L3, dW decreased but remained over 0.3 (Fig. 5(C)).With L4, the weights approached to the results of the correlation learning in the early phase then dW increased upto about 0.3 (Fig. 5(D)).With L5, the weights approached to the correlation learning results and stayed close to it but maintained small difference (Fig. 5(E)).
In the retrieval stage, input patters with different levels of errors (Fig. 3(B)) were presented to our network for 120ms by adding pulse currents which have the same parameter setting to the one in the learning stage.Here, 120ms (5 spikes) allowed our network to receive sufficient amount of stimulus input.The time length of the retrieval stage was 800ms, which was sufficiently long for silicon neurons to reach stable states.We calculated overlap index to evaluate the similarity between the state of silicon neurons and the stored patterns.The overlap   between the state of silicon neurons and the u-th stored pattern is obtained by the following equations .

𝑀 𝑢 (𝑡
where N is the number of silicon neurons,    represents the j-th pixel in the u-th stored patterns with value of 1 or -1 that correspond to black or white, respectively.Function   () is the phase value of the j-th neuron, which is defined by Eq. (17).And    is the start time of the k-th spike for the j-th silicon neuron.When the network successfully retrieves the stored pattern, the relevant overlap equals to 1. Figure 6 shows the raster plot (A) and overlaps (B) when our silicon neuronal network is trained by the L5 rule in the learning stage and the pattern with 20% errors in Fig. 3(B) was applied in the retrieval stage.The input pattern was observed for 5 spikes firstly and then stored pattern 1 and its reversed pattern alternately appeared from 0.4s (Fig. 6(A)).Then the overlap  1 reached to 1 when the stored pattern 1 was retrieved in the network (Fig. 6(B)).We calculated the retrieval probability in the auto-associative memory when our silicon neuronal network is trained by the Hebbian learning rules (L1)-(L5) and the correlation learning.Totally 100 input patterns were applied; 10 patterns with 10 different error rates from 5% to 50% by 5% step.We counted the success of memory retrieval if M u reached 1 at the end of the retrieval stage (Fig. 7).The L1-3 rules could not retrieve at all.The L4 and L5 rules had better performance than the correlation learning rule when the error rate was larger than 15%.The L5 rule showed higher retrieval probability than L4 rule when input pattern contained 35% errors.

Conclusions
The spike-time-dependent Hebbian learning modifies the synaptic weights according to the difference of spike timing between pre-and post-synaptic neurons.Our silicon neuronal network succeeded to execute auto-associative memory based on this learning concept.
We trained our silicon neuronal network with 5 types of Hebbian learning rules and evaluated them by the auto-associative memory.The simulation results suggest that the AC Hebbian learning rules (L4, L5) boost the performance of the auto-associative memory in our network in comparison to the correlation learning.The difference of performance between L4 and L5 is not large in comparison to this boost, and we expect it is brought by a kind of distortion of ∆W over the distribution of the weights caused by the soft bounding terms.This will to be analyzed in detail in our future works.Our silicon neuronal network model is optimized for implementation by digital arithmetic circuits.Moreover, many effects were made in the circuit design to reduce the number of LUTs for the Hebbian Time (s) Time (s) learning module.The system is estimated to be able to run at up to 97.972MHz, which realizes over 35 times faster than real-time of the nerve system.
In the future, we will complete our VHDL design and download our silicon neuronal network to an FPGA device and verify its functionality.The spike-timing-dependent plasticity (STDP) with asymmetric learning curves is another popular learning rule that modifies the connection weights depending on the difference between the spike timings of pre-and post-synaptic neurons.We will evaluate the performance of our silicon neuronal network trained with this rule by the spatio-temoporal memory.We will also expand our network to 1000 neurons to enhance the amount of the storable data.Because the sequential aspect of our silicon neuronal network model is increased by larger number of silicon neurons in a network, the calculation speed in FPGA devices will be reduced.Though, we expect we can keep about 35 times faster than real-time by exploiting parallelism of the FPGA circuits.

Fig. 3 .
Fig. 3. Stored patterns (A) and a set of input patterns (B) generated by randomly reversing pixels of stored pattern 1.

Fig. 6 .
Fig. 6.Raster plot (50 out of 256 neurons are displayed) and overlap of the memory retrieval in the Class I mode network trained by the L5 rule.The input pattern includes 20% errors shown in Fig. 3(B).

Fig. 7 .
Fig. 7. Retrieval probability with the correlation learning rule and the Hebbian learning rules.

Table 1 .
Table 1 lists the device utilization of the circuit synthesized by the Xilinx ISE tool.The fixed point number, shift operation for avoiding multiplier and LUTs for exponential functions are adopted to reduce the hardware utilization.The fast speed of our network can reach up to 97.972MHz according to the synthesize report.Device utilization on FPGA device 3, 24-28,  2008