Connectionists: 2 new papers on Hebbian learning

[padams@notes.cc.sunysb.edu]

Dear Colleagues - Recently, it has been shown that biological Hebbian 
plasticity (e.g. ltp) is not completely synapse-specific. We would like to 
draw your attention to our 2 new papers examining the effect of such minor 
inaccuracy in 2 simple neural net learning paradigms, PCA and ICA. For the 
linear (PCA) learning rule, inaccuracy produces graceful degradation, but 
for  nonlinear (ICA) rules it can have a catastrophic effect. We propose 
that a critical learning problem in the brain is achieving the necessary 
(but likely synaptically impossible) extraordinary accuracy. Citations and 
abstracts follow.

1. Radulescu, A., Cox, K., and Adams, P. (2009). Hebbian errors in 
learning: an analysis using the Oja model. J. Theor. Biol. 258, 489-501

"Recent work on long term potentiation in brain slices shows that Hebb's 
rule is not completely synapse-specific, probably due to intersynapse 
diffusion of calcium or other factors. We previously suggested that such 
errors in Hebbian learning might be analogous to mutations in evolution. 
We examine this proposal quantitatively, extending the classical Oja 
unsupervised model of learning by a single linear neuron to include 
Hebbian inspecificity. We introduce an error matrix E, which expresses 
possible crosstalk between updating at different connections. When there 
is no inspecificity, this gives the classical result of convergence to the 
first principal component of the input distribution (PC1). We show the 
modified algorithm converges to the leading eigenvector of the matrix EC, 
where C is the input covariance matrix. In the most biologically plausible 
case when there are no intrinsically privileged connections, E has 
diagonal elements Q and off-diagonal elements (1-Q)/(n-1), where Q, the 
quality, is expected to decrease with the number of inputs n and with a 
synaptic parameter b that reflects synapse density, calcium diffusion, 
etc. We study the dependence of the learning accuracy on b, n and the 
amount of input activity or correlation (analytically and 
computationally). We find that accuracy decreases (learning becomes 
gradually less useful) with increases in b, particularly for intermediate 
(i.e., biologically realistic) correlation strength, although some useful 
learning always occurs up to the trivial limit Q=1/n. We discuss the 
relation of our results to Hebbian unsupervised learning in the brain. 
When the mechanism lacks specificity, the network fails to learn the 
expected, and typically most useful, result, especially when the input 
correlation is weak. Hebbian crosstalk would reflect the very high density 
of synapses along dendrites, and inevitably degrades learning." 

link: http://dx.doi.org/10.1016/j.jtbi.2009.01.036

2     Cox K.J.A.  and Adams P.R. (2009) Hebbian crosstalk prevents 
nonlinear unsupervised learning. Front. Comput. Neurosci. 3:1-20 2009

 "Learning is thought to occur by localized, activity-induced changes in 
the strength of synaptic connections between neurons. Recent work has 
shown that activity-dependent changes at one connection can affect changes 
at others ("crosstalk"). We studied the role of such crosstalk in 
nonlinear Hebbian learning using a neural network implementation of 
Independent Components Analysis (ICA). We find that there is a sudden 
qualitative change in the performance of the network at a threshold 
crosstalk level and discuss the implications of this for nonlinear 
learning from higher-order correlations in the neocortex." 

link: 
http://frontiersin.org/computationalneuroscience/paper/10.3389/neuro.10/011.2009/

Comments and feedback welcome 
- Paul Adams and Kingsley Cox

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20091018/b30b6244/attachment.html