eprop_iaf_adapt_bsshslm_2020 – Current-based leaky integrate-and-fire neuron model with delta-shaped postsynaptic currents and threshold adaptation for e-prop plasticity

Description

eprop_iaf_adapt_bsshslm_2020 is an implementation of a leaky integrate-and-fire neuron model with delta-shaped postsynaptic currents and threshold adaptation used for eligibility propagation (e-prop) plasticity.

E-prop plasticity was originally introduced and implemented in TensorFlow in [1].

The suffix _bsshslm_2020 follows the NEST convention to indicate in the model name the paper that introduced it by the first letter of the authors’ last names and the publication year.

Note

The neuron dynamics of the eprop_iaf_adapt_bsshslm_2020 model (excluding e-prop plasticity and the threshold adaptation) are similar to the neuron dynamics of the iaf_psc_delta model, with minor differences, such as the propagator of the post-synaptic current and the voltage reset upon a spike.

The membrane voltage time course \(v_j^t\) of the neuron \(j\) is given by:

\[\begin{split}v_j^t &= \alpha v_j^{t-1}+\sum_{i \neq j}W_{ji}^\mathrm{rec}z_i^{t-1} + \sum_i W_{ji}^\mathrm{in}x_i^t-z_j^{t-1}v_\mathrm{th} \,, \\ \alpha &= e^{-\frac{\Delta t}{\tau_\mathrm{m}}} \,,\end{split}\]

whereby \(W_{ji}^\mathrm{rec}\) and \(W_{ji}^\mathrm{in}\) are the recurrent and input synaptic weights, and \(z_i^{t-1}\) and \(x_i^t\) are the recurrent and input presynaptic spike state variables, respectively.

Descriptions of further parameters and variables can be found in the table below.

The threshold adaptation is given by:

\[\begin{split}A_j^t &= v_\mathrm{th} + \beta a_j^t \,, \\ a_j^t &= \rho a_j^{t-1} + z_j^{t-1} \,, \\ \rho &= e^{-\frac{\Delta t}{\tau_\mathrm{a}}} \,.\end{split}\]

The spike state variable is expressed by a Heaviside function:

\[z_j^t = H\left(v_j^t-A_j^t\right) \,.\]

If the membrane voltage crosses the adaptive threshold voltage \(A_j^t\), a spike is emitted and the membrane voltage is reduced by \(v_\text{th}\) in the next time step. After the time step of the spike emission, the neuron is not able to spike for an absolute refractory period \(t_\text{ref}\).

An additional state variable and the corresponding differential equation represents a piecewise constant external current.

Furthermore, the pseudo derivative of the membrane voltage needed for e-prop plasticity is calculated:

\[\psi_j^t = \frac{\gamma}{v_\text{th}} \text{max} \left(0, 1-\left| \frac{v_j^t-A_j^t}{v_\text{th}}\right| \right) \,.\]

See the documentation on the iaf_psc_delta neuron model for more information on the integration of the subthreshold dynamics.

The change of the synaptic weight is calculated from the gradient \(g\) of the loss \(E\) with respect to the synaptic weight \(W_{ji}\): \(\frac{\mathrm{d}{E}}{\mathrm{d}{W_{ij}}}=g\) which depends on the presynaptic spikes \(z_i^{t-1}\), the surrogate-gradient / pseudo-derivative of the postsynaptic membrane voltage \(\psi_j^t\) (which together form the eligibility trace \(e_{ji}^t\)), and the learning signal \(L_j^t\) emitted by the readout neurons.

\[\begin{split}\frac{\mathrm{d}E}{\mathrm{d}W_{ji}} = g &= \sum_t L_j^t \bar{e}_{ji}^t, \\ e_{ji}^t &= \psi_j^t \left(\bar{z}_i^{t-1} - \beta \epsilon_{ji,a}^{t-1}\right)\,, \\ \epsilon^{t-1}_{ji,\text{a}} &= \psi_j^{t-1}\bar{z}_i^{t-2} + \left( \rho - \psi_j^{t-1} \beta \right) \epsilon^{t-2}_{ji,a}\,. \\\end{split}\]

The eligibility trace and the presynaptic spike trains are low-pass filtered with some exponential kernels:

\[\begin{split}\bar{e}_{ji}^t&=\mathcal{F}_\kappa(e_{ji}^t) \;\text{with}\, \kappa=e^{-\frac{\Delta t}{ \tau_\text{m,out}}}\,,\\ \bar{z}_i^t&=\mathcal{F}_\alpha(z_i^t)\,,\\ \mathcal{F}_\alpha(z_i^t) &= \alpha\, \mathcal{F}_\alpha(z_i^{t-1}) + z_i^t \;\text{with}\, \mathcal{F}_\alpha(z_i^0)=z_i^0\,\,,\end{split}\]

whereby \(\tau_\text{m,out}\) is the membrane time constant of the readout neuron.

Furthermore, a firing rate regularization mechanism keeps the average firing rate \(f^\text{av}_j\) of the postsynaptic neuron close to a target firing rate \(f^\text{target}\). The gradient \(g^\text{reg}\) of the regularization loss \(E^\text{reg}\) with respect to the synaptic weight \(W_{ji}\) is given by:

\[\frac{\mathrm{d}E^\text{reg}}{\mathrm{d}W_{ji}} = g^\text{reg} = c_\text{reg} \sum_t \frac{1}{Tn_\text{trial}} \left( f^\text{target}-f^\text{av}_j\right)e_{ji}^t\,,\]

whereby \(c_\text{reg}\) scales the overall regularization and the average is taken over the time that passed since the previous update, that is, the number of trials \(n_\text{trial}\) times the duration of an update interval \(T\).

The overall gradient is given by the addition of the two gradients.

For more information on e-prop plasticity, see the documentation on the other e-prop models:

• eprop_iaf_bsshslm_2020

• eprop_readout_bsshslm_2020

• eprop_synapse_bsshslm_2020

• eprop_learning_signal_connection_bsshslm_2020

Details on the event-based NEST implementation of e-prop can be found in [2].

Parameters

The following parameters can be set in the status dictionary.

Neuron parameters
Parameter	Unit	Math equivalent	Default	Description
adapt_beta		\(\beta\)	1.0	Prefactor of the threshold adaptation
adapt_tau	ms	\(\tau_\text{a}\)	10.0	Time constant of the threshold adaptation
C_m	pF	\(C_\text{m}\)	250.0	Capacitance of the membrane
c_reg		\(c_\text{reg}\)	0.0	Prefactor of firing rate regularization
E_L	mV	\(E_\text{L}\)	-70.0	Leak / resting membrane potential
f_target	Hz	\(f^\text{target}\)	10.0	Target firing rate of rate regularization
gamma		\(\gamma\)	0.3	Scaling of surrogate gradient / pseudo-derivative of membrane voltage
I_e	pA	\(I_\text{e}\)	0.0	Constant external input current
regular_spike_arrival	Boolean		True	If True, the input spikes arrive at the end of the time step, if False at the beginning (determines PSC scale)
surrogate_gradient_function		\(\psi\)	piecewise_linear	Surrogate gradient / pseudo-derivative function [“piecewise_linear”]
t_ref	ms	\(t_\text{ref}\)	2.0	Duration of the refractory period
tau_m	ms	\(\tau_\text{m}\)	10.0	Time constant of the membrane
V_min	mV	\(v_\text{min}\)	-1.79e+308	Absolute lower bound of the membrane voltage
V_th	mV	\(v_\text{th}\)	-55.0	Spike threshold voltage

The following state variables evolve during simulation.

Neuron state variables and recordables
State variable	Unit	Math equivalent	Initial value	Description
adaptation		\(a_j\)	0.0	Adaptation variable
learning_signal		\(L_j\)	0.0	Learning signal
surrogate_gradient		\(\psi_j\)	0.0	Surrogate gradient
V_m	mV	\(v_j\)	-70.0	Membrane voltage
V_th_adapt	mV	\(A_j\)	-55.0	Adapting spike threshold

Recordables

The following variables can be recorded:

• adaptation variable adaptation

• adapting spike threshold V_th_adapt

• learning signal learning_signal

• membrane potential V_m

• surrogate gradient surrogate_gradient

Usage

This model can only be used in combination with the other e-prop models, whereby the network architecture requires specific wiring, input, and output. The usage is demonstrated in several supervised regression and classification tasks reproducing among others the original proof-of-concept tasks in [1].

References

[1] (1,2)

Bellec G, Scherr F, Subramoney F, Hajek E, Salaj D, Legenstein R, Maass W (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11:3625. https://doi.org/10.1038/s41467-020-17236-y

[2]

Korcsak-Gorzo A, Stapmanns J, Espinoza Valverde JA, Dahmen D, van Albada SJ, Bolten M, Diesmann M. Event-based implementation of eligibility propagation (in preparation)

Sends

SpikeEvent

Receives

SpikeEvent, CurrentEvent, LearningSignalConnectionEvent, DataLoggingRequest

See also

Examples using this model

• Tutorial on learning to accumulate evidence with e-prop

• Tutorial on learning to generate an infinite loop with e-prop

• Tutorial on learning to generate handwritten text with e-prop