eprop_iaf_bsshslm_2020 – Current-based leaky integrate-and-fire neuron model with delta-shaped postsynaptic currents for e-prop plasticity ========================================================================================================================================== Description +++++++++++ ``eprop_iaf_bsshslm_2020`` is an implementation of a leaky integrate-and-fire neuron model with delta-shaped postsynaptic currents used for eligibility propagation (e-prop) plasticity. E-prop plasticity was originally introduced and implemented in TensorFlow in [1]_. The suffix ``_bsshslm_2020`` follows the NEST convention to indicate in the model name the paper that introduced it by the first letter of the authors' last names and the publication year. .. note:: The neuron dynamics of the ``eprop_iaf_bsshslm_2020`` model (excluding e-prop plasticity) are similar to the neuron dynamics of the ``iaf_psc_delta`` model, with minor differences, such as the propagator of the post-synaptic current and the voltage reset upon a spike. The membrane voltage time course :math:`v_j^t` of the neuron :math:`j` is given by: .. math:: v_j^t &= \alpha v_j^{t-1}+\sum_{i \neq j}W_{ji}^\mathrm{rec}z_i^{t-1} + \sum_i W_{ji}^\mathrm{in}x_i^t-z_j^{t-1}v_\mathrm{th} \,, \\ \alpha &= e^{-\frac{\Delta t}{\tau_\mathrm{m}}} \,, whereby :math:`W_{ji}^\mathrm{rec}` and :math:`W_{ji}^\mathrm{in}` are the recurrent and input synaptic weights, and :math:`z_i^{t-1}` and :math:`x_i^t` are the recurrent and input presynaptic spike state variables, respectively. Descriptions of further parameters and variables can be found in the table below. The spike state variable is expressed by a Heaviside function: .. math:: z_j^t = H\left(v_j^t-v_\mathrm{th}\right) \,. If the membrane voltage crosses the threshold voltage :math:`v_\text{th}`, a spike is emitted and the membrane voltage is reduced by :math:`v_\text{th}` in the next time step. After the time step of the spike emission, the neuron is not able to spike for an absolute refractory period :math:`t_\text{ref}`. An additional state variable and the corresponding differential equation represents a piecewise constant external current. Furthermore, the pseudo derivative of the membrane voltage needed for e-prop plasticity is calculated: .. math:: \psi_j^t = \frac{\gamma}{v_\text{th}} \text{max} \left(0, 1-\left| \frac{v_j^t-v_\mathrm{th}}{v_\text{th}}\right| \right) \,. See the documentation on the ``iaf_psc_delta`` neuron model for more information on the integration of the subthreshold dynamics. The change of the synaptic weight is calculated from the gradient :math:`g` of the loss :math:`E` with respect to the synaptic weight :math:`W_{ji}`: :math:`\frac{\mathrm{d}{E}}{\mathrm{d}{W_{ij}}}=g` which depends on the presynaptic spikes :math:`z_i^{t-1}`, the surrogate-gradient / pseudo-derivative of the postsynaptic membrane voltage :math:`\psi_j^t` (which together form the eligibility trace :math:`e_{ji}^t`), and the learning signal :math:`L_j^t` emitted by the readout neurons. .. math:: \frac{\mathrm{d}E}{\mathrm{d}W_{ji}} = g &= \sum_t L_j^t \bar{e}_{ji}^t, \\ e_{ji}^t &= \psi^t_j \bar{z}_i^{t-1}\,, \\ The eligibility trace and the presynaptic spike trains are low-pass filtered with some exponential kernels: .. math:: \bar{e}_{ji}^t &= \mathcal{F}_\kappa(e_{ji}^t) \;\text{with}\, \kappa=e^{-\frac{\Delta t}{ \tau_\text{m,out}}}\,,\\ \bar{z}_i^t&=\mathcal{F}_\alpha(z_i^t)\,,\\ \mathcal{F}_\alpha(z_i^t) &= \alpha\, \mathcal{F}_\alpha(z_i^{t-1}) + z_i^t \;\text{with}\, \mathcal{F}_\alpha(z_i^0)=z_i^0\,, whereby :math:`\tau_\text{m,out}` is the membrane time constant of the readout neuron. Furthermore, a firing rate regularization mechanism keeps the average firing rate :math:`f^\text{av}_j` of the postsynaptic neuron close to a target firing rate :math:`f^\text{target}`. The gradient :math:`g^\text{reg}` of the regularization loss :math:`E^\text{reg}` with respect to the synaptic weight :math:`W_{ji}` is given by: .. math:: \frac{\mathrm{d}E^\text{reg}}{\mathrm{d}W_{ji}} = g^\text{reg} = c_\text{reg} \sum_t \frac{1}{Tn_\text{trial}} \left( f^\text{target}-f^\text{av}_j\right)e_{ji}^t\,, whereby :math:`c_\text{reg}` scales the overall regularization and the average is taken over the time that passed since the previous update, that is, the number of trials :math:`n_\text{trial}` times the duration of an update interval :math:`T`. The overall gradient is given by the addition of the two gradients. For more information on e-prop plasticity, see the documentation on the other e-prop models: * :doc:`eprop_iaf_adapt_bsshslm_2020<../models/eprop_iaf_adapt_bsshslm_2020/>` * :doc:`eprop_readout_bsshslm_2020<../models/eprop_readout_bsshslm_2020/>` * :doc:`eprop_synapse_bsshslm_2020<../models/eprop_synapse_bsshslm_2020/>` * :doc:`eprop_learning_signal_connection_bsshslm_2020<../models/eprop_learning_signal_connection_bsshslm_2020/>` Details on the event-based NEST implementation of e-prop can be found in [2]_. Parameters ++++++++++ The following parameters can be set in the status dictionary. =========================== ======= ======================= ================ =================================== **Neuron parameters** ---------------------------------------------------------------------------------------------------------------- Parameter Unit Math equivalent Default Description =========================== ======= ======================= ================ =================================== C_m pF :math:`C_\text{m}` 250.0 Capacitance of the membrane c_reg :math:`c_\text{reg}` 0.0 Prefactor of firing rate regularization E_L mV :math:`E_\text{L}` -70.0 Leak / resting membrane potential f_target Hz :math:`f^\text{target}` 10.0 Target firing rate of rate regularization gamma :math:`\gamma` 0.3 Scaling of surrogate gradient / pseudo-derivative of membrane voltage I_e pA :math:`I_\text{e}` 0.0 Constant external input current regular_spike_arrival Boolean True If True, the input spikes arrive at the end of the time step, if False at the beginning (determines PSC scale) surrogate_gradient_function :math:`\psi` piecewise_linear Surrogate gradient / pseudo-derivative function ["piecewise_linear"] t_ref ms :math:`t_\text{ref}` 2.0 Duration of the refractory period tau_m ms :math:`\tau_\text{m}` 10.0 Time constant of the membrane V_min mV :math:`v_\text{min}` -1.79e+308 Absolute lower bound of the membrane voltage V_th mV :math:`v_\text{th}` -55.0 Spike threshold voltage =========================== ======= ======================= ================ =================================== The following state variables evolve during simulation. ================== ==== =============== ============= ========================================================== **Neuron state variables and recordables** ---------------------------------------------------------------------------------------------------------------- State variable Unit Math equivalent Initial value Description ================== ==== =============== ============= ========================================================== learning_signal pA :math:`L_j` 0.0 Learning signal surrogate_gradient :math:`\psi_j` 0.0 Surrogate gradient / pseudo-derivative of membrane voltage V_m mV :math:`v_j` -70.0 Membrane voltage ================== ==== =============== ============= ========================================================== Recordables +++++++++++ The following variables can be recorded: - learning signal ``learning_signal`` - membrane potential ``V_m`` - surrogate gradient ``surrogate_gradient`` Usage +++++ This model can only be used in combination with the other e-prop models, whereby the network architecture requires specific wiring, input, and output. The usage is demonstrated in several :doc:`supervised regression and classification tasks <../auto_examples/eprop_plasticity/index>` reproducing among others the original proof-of-concept tasks in [1]_. References ++++++++++ .. [1] Bellec G, Scherr F, Subramoney F, Hajek E, Salaj D, Legenstein R, Maass W (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11:3625. https://doi.org/10.1038/s41467-020-17236-y .. [2] Korcsak-Gorzo A, Stapmanns J, Espinoza Valverde JA, Dahmen D, van Albada SJ, Bolten M, Diesmann M. Event-based implementation of eligibility propagation (in preparation) Sends ++++++++ SpikeEvent Receives ++++++++ SpikeEvent, CurrentEvent, LearningSignalConnectionEvent, DataLoggingRequest See also ++++++++ :doc:`Neuron `, :doc:`E-Prop Plasticity `, :doc:`Current-Based `, :doc:`Integrate-And-Fire ` Examples using this model ++++++++++++++++++++++++++ .. listexamples:: eprop_iaf_bsshslm_2020