eprop_readout_bsshslm_2020 – Current-based leaky integrate readout neuron model with delta-shaped or exponentially filtered postsynaptic currents for e-prop plasticity

Description

eprop_readout_bsshslm_2020 is an implementation of an integrate-and-fire neuron model with delta-shaped postsynaptic currents used as readout neuron for eligibility propagation (e-prop) plasticity.

E-prop plasticity was originally introduced and implemented in TensorFlow in [1].

The suffix _bsshslm_2020 follows the NEST convention to indicate in the model name the paper that introduced it by the first letter of the authors’ last names and the publication year.

The membrane voltage time course \(v_j^t\) of the neuron \(j\) is given by:

\[\begin{split}v_j^t &= \kappa v_j^{t-1} + \zeta \sum_{i \neq j} W_{ji}^\text{out} z_i^{t-1} \,, \\ \kappa &= e^{ -\frac{ \Delta t }{ \tau_\text{m} } } \,, \\ \zeta &= \begin{cases} 1 \\ 1 - \kappa \end{cases} \,, \\\end{split}\]

where \(W_{ji}^\text{out}\) is the output synaptic weight matrix and \(z_i^{t-1}\) is the recurrent presynaptic spike state variable.

Descriptions of further parameters and variables can be found in the table below.

The spike state variable of a presynaptic neuron is expressed by a Heaviside function:

\[\begin{split}z_i^t = H \left( v_i^t - v_\text{th} \right) \,. \\\end{split}\]

An additional state variable and the corresponding differential equation represents a piecewise constant external current.

See the documentation on the iaf_psc_delta neuron model for more information on the integration of the subthreshold dynamics.

The change of the synaptic weight is calculated from the gradient \(g\) of the loss \(E\) with respect to the synaptic weight \(W_{ji}\): \(\frac{ \text{d}E }{ \text{d} W_{ij} }\) which depends on the presynaptic spikes \(z_i^{t-1}\) and the learning signal \(L_j^t\) emitted by the readout neurons.

\[\begin{split}\frac{ \text{d} E }{ \text{d} W_{ji} } = \sum_t L_j^t \bar{z}_i^{t-1} \,. \\\end{split}\]

The presynaptic spike trains are low-pass filtered with the following exponential kernel:

\[\begin{split}\bar{z}_i^t &=\mathcal{F}_\kappa(z_i^t) \,, \\ \mathcal{F}_\kappa(z_i^t) &= \kappa \mathcal{F}_\kappa \left( z_i^{t-1} \right) + z_i^t \,, \\ \mathcal{F}_\kappa(z_i^0) &= z_i^0 \,. \\\end{split}\]

Since readout neurons are leaky integrators without a spiking mechanism, the formula for computing the gradient lacks the surrogate gradient / pseudo-derivative and a firing regularization term.

The learning signal \(L_j^t\) is given by the non-plastic feedback weight matrix \(B_{jk}\) and the continuous error signal \(e_k^t\) emitted by readout neuron \(k\):

\[\begin{split}L_j^t = B_{jk} e_k^t \,. \\\end{split}\]

The error signal depends on the selected loss function. If a mean squared error loss is selected, then:

\[\begin{split}e_k^t = y_k^t - y_k^{*,t} \,, \\\end{split}\]

where the readout signal \(y_k^t\) corresponds to the membrane voltage of readout neuron \(k\) and \(y_k^{*,t}\) is the real-valued target signal.

If a cross-entropy loss is selected, then:

\[\begin{split}e^k_t &= \pi_k^t - \pi_k^{*,t} \,, \\ \pi_k^t &= \text{softmax}_k \left( y_1^t, ..., y_K^t \right) = \frac{ \exp \left( y_k^t\right) }{ \sum_{k'} \exp \left( y_{k'}^t \right) } \,, \\\end{split}\]

where the readout signal \(\pi_k^t\) corresponds to the softmax of the membrane voltage of readout neuron \(k\) and \(\pi_k^{*,t}\) is the one-hot encoded target signal.

Furthermore, the readout and target signal are zero before the onset of the learning window in each update interval.

For more information on e-prop plasticity, see the documentation on the other e-prop models:

• eprop_iaf_bsshslm_2020

• eprop_iaf_adapt_bsshslm_2020

• eprop_synapse_bsshslm_2020

• eprop_learning_signal_connection_bsshslm_2020

Details on the event-based NEST implementation of e-prop can be found in [2].

Parameters

The following parameters can be set in the status dictionary.

Neuron parameters
Parameter	Unit	Math equivalent	Default	Description
C_m	pF	\(C_\text{m}\)	250.0	Capacitance of the membrane
E_L	mV	\(E_\text{L}\)	0.0	Leak / resting membrane potential
I_e	pA	\(I_\text{e}\)	0.0	Constant external input current
regular_spike_arrival	Boolean		True	If True, the input spikes arrive at the end of the time step, if False at the beginning (determines PSC scale)
tau_m	ms	\(\tau_\text{m}\)	10.0	Time constant of the membrane
V_min	mV	\(v_\text{min}\)	negative maximum value representable by a double type in C++	Absolute lower bound of the membrane voltage

E-prop parameters
Parameter	Unit	Math equivalent	Default	Description
loss		\(E\)	“mean_squared_error”	Loss function [“mean_squared_error”, “cross_entropy”]

Recordables

The following state variables evolve during simulation and can be recorded.

Neuron state variables and recordables
State variable	Unit	Math equivalent	Initial value	Description
V_m	mV	\(v_j\)	0.0	Membrane voltage

E-prop state variables and recordables
State variable	Unit	Math equivalent	Initial value	Description
error_signal	mV	\(L_j\)	0.0	Error signal
readout_signal	mV	\(y_j\)	0.0	Readout signal
readout_signal_unnorm	mV		0.0	Unnormalized readout signal
target_signal	mV	\(y^*_j\)	0.0	Target signal

Usage

This model can only be used in combination with the other e-prop models and the network architecture requires specific wiring, input, and output. The usage is demonstrated in several supervised regression and classification tasks reproducing among others the original proof-of-concept tasks in [1].

References

[1] (1,2)

Bellec G, Scherr F, Subramoney F, Hajek E, Salaj D, Legenstein R, Maass W (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11:3625. https://doi.org/10.1038/s41467-020-17236-y

[2]

Korcsak-Gorzo A, Stapmanns J, Espinoza Valverde JA, Plesser HE, Dahmen D, Bolten M, Van Albada SJ, Diesmann M. Event-based implementation of eligibility propagation (in preparation)

Sends

LearningSignalConnectionEvent, DelayedRateConnectionEvent

Receives

SpikeEvent, CurrentEvent, DelayedRateConnectionEvent, DataLoggingRequest

See also

Examples using this model

• Tutorial on learning to accumulate evidence with e-prop after Bellec et al. (2020)

• Tutorial on learning to generate a lemniscate with e-prop after Bellec et al. (2020)

• Tutorial on learning to generate handwritten text with e-prop after Bellec et al. (2020)

• Tutorial on learning to generate sine waves with e-prop after Bellec et al. (2020)