Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Tony Prescott]

Dear colleagues,

The Clune et al. article we are discussing mentions that selection for
reduced connectivity could be a "spandrel" (the consequence of
selection for something else) but does not explore this possibility in
much depth.  In the case of biological brains it is hard to see why
low connectivity should be directly selected rather than arising
through the need to keep a lid on the size and metabolic cost of
maintaining the brain.  A 1991 paper by Ringo
(http://www.ncbi.nlm.nih.gov/pubmed/1657274) shows that larger brains
cannot maintain the same degree of inter-connectedness as smaller ones
and therefore long-range connections are necessary sparser if
increased an in neuron count is not going to give rise to an
exponential increase in brain size.  Reduced connectivity is therefore
an architectural constraint for larger brains in not too dissimilar
way to the need for spandrels in cathedral domes (as discussed by
Gould, 1979).

An important consideration for biological brains is connection length.
 Leise 1990 (http://www.ncbi.nlm.nih.gov/pubmed/2194614) provides a
useful summary of the reasons why nervous systems are composed of
physically modular components with a high number of short-range
connections and low number of longer range ones.  As the literature on
small world networks show, however, it is important not to assume that
physical modularity requires functional modularity.  Appropriate
sparse connectivity can allow fast communication and synchronisation
across large  networks that can support distributed functional
modules.

Regards,

Tony Prescott



On 24 February 2013 03:49, Terry Sejnowski <terry at salk.edu> wrote:
> G. Mitchison, Neuronal branching patterns and the economy of cortical wiring, Proc. Roy. Soc. London
> B Biol. Sci. 245 (1991) 151{158
>
> D.B. Chklovskii, C.F. Stevens, Wiring optimization in the brain, Neural Information Processing Systems
> (1999)
>
> Koulakov AA, Chklovskii DB. Orientation preference patterns in mammalian visual cortex: a wire length minimization approach.  Neuron. 2001 Feb;29(2):519-27.
>
> Chklovskii DB, Schikorski T, Stevens CF. Wiring optimization in cortical circuits.
> Neuron. 2002 Apr 25;34(3):341-7.
>
> Terry
>
> -----
>
>> The paper mentions that Santiago Ram<F3>n y Cajal already pointed out
>> that evolution has created mostly short connections in animal brains.
>>
>> Minimization of connection costs should also encourage modularization,
>> e.g., http://arxiv.org/abs/1210.0118 (2012).
>>
>> But who first had such a wire length term in an objective function to
>> be minimized by evolutionary computation or other machine learning
>> methods?
>> I am aware of pioneering work by Legenstein and Maass:
>>
>> R. A. Legenstein and W. Maass. Neural circuits for pattern recognition
>> with small total wire length. Theoretical Computer Science,
>> 287:239-249, 2002.
>> R. A. Legenstein and W. Maass. Wire length as a circuit complexity
>> measure. Journal of Computer and System Sciences, 70:53-72, 2005.
>>
>> Is there any earlier relevant work? Pointers will be appreciated.
>>
>> Juergen Schmidhuber
>> http://www.idsia.ch/~juergen/whatsnew.html



-- 
------------------------------------------------------------------------------------------------------------------------------------------------------
Consider submitting to Living Machines II
(http://csnetwork.eu/livingmachines/conf2013)
The 2nd International Conference on Biomimetic and Biohybrid Systems,
29th July to 2nd August 2013, Natural History Museum, London.
Deadline for papers March 15th, 2013.
------------------------------------------------------------------------------------------------------------------------------------------------------

Tony J Prescott, Professor of Cognitive Neuroscience
Department of Psychology, Western Bank, Sheffield, S10 2TN, United Kingdom.
email: t.j.prescott at sheffield.ac.uk, skype: tonyjprescott, twitter: shefrobotics
phone: +44 114 2226547, fax: +44 114 2766515
http://www.shef.ac.uk/psychology/staff/academic/tony-prescott
http://www.abrg.group.shef.ac.uk/people/tony/

Director, Sheffield Centre for Robotics (SCentRo) (http://www.scentro.ac.uk/)
Director, Active Touch Laboratory (ATL at S)
(http://www.shef.ac.uk/psychology/research/groups/atlas)
Co-director, Adaptive Behaviour Research Group (ABRG)
(http://www.abrg.group.shef.ac.uk/)
Co-Chair, Living Machines 2013 (http://csnetwork.eu/livingmachines/conf2013)
------------------------------------------------------------------------------------------------------------------------------------------------------