Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Don Mathis]

I'm sure Jurgen remembers this paper:

http://dl.acm.org/citation.cfm?id=1326963

Computational consequences of a bias toward short connections

Authors:	Robert A. Jacobs	
Michael I. Jordan	Massachusetts Institute of Technology
Published in:
Journal of Cognitive Neuroscience
Volume 4 Issue 4, Fall 1992 
Pages 323-336 
MIT Press Cambridge, MA, USA 
doi>10.1162/jocn.1992.4.4.323

------------------------------------------------------------
Donald W. Mathis
Johns Hopkins University
Cognitive Science Department
3400 N. Charles Street
Krieger Hall 141A
Baltimore, MD 21218
Tel: 410-516-6851
Fax: 410-516-8020
mathis at jhu.edu
http://cogsci.jhu.edu
 

On Feb 22, 2013, at 4:18 PM, Richard Loosemore <rloosemore at susaro.com>
 wrote:

> 
> I hate to say this, but during discussions with fellow students back in 1987, I remember pointing out that it was not terribly surprising that the cortex consisted of columns (i.e. modules) with dense internal connectivity, with less-dense connections between columns -- not surprising, because the alternative was to try to make the brain less modular and connect every neuron in each column to all the neurons in all the other columns, and the result would be brains that were a million times larger than they are (due to all the extra wiring).
> 
> The same logic applies in all systems where it is costly to connect every element to every other: the optimal connectivity is well-connected, tightly clustered groups of elements.
> 
> During those discussions the point was considered so obvious that it sparked little comment. Ever since then I have told students in my lectures that this would be the evolutionary reason for cortical columns to exist.
> 
> So I am a little confused now. Can someone explain what I am missing .........?
> 
> Richard Loosemore
> Department of Physical and Mathematical Sciences,
> Wells College
> 
> 
> 
> On 2/13/13 9:48 AM, Juergen Schmidhuber wrote:
>> The paper mentions that Santiago Ramón y Cajal already pointed out that evolution has created mostly short connections in animal brains.
>> 
>> Minimization of connection costs should also encourage modularization, e.g., http://arxiv.org/abs/1210.0118 (2012).
>> 
>> But who first had such a wire length term in an objective function to be minimized by evolutionary computation or other machine learning methods?
>> I am aware of pioneering work by Legenstein and Maass:
>> 
>> R. A. Legenstein and W. Maass. Neural circuits for pattern recognition with small total wire length. Theoretical Computer Science, 287:239-249, 2002.
>> R. A. Legenstein and W. Maass. Wire length as a circuit complexity measure. Journal of Computer and System Sciences, 70:53-72, 2005.
>> 
>> Is there any earlier relevant work? Pointers will be appreciated.
>> 
>> Jürgen Schmidhuber
>> http://www.idsia.ch/~juergen/whatsnew.html
>> 
>> 
>> 
>> 
>> On Feb 10, 2013, at 3:14 AM, Jeff Clune wrote:
>> 
>>> Hello all,
>>> 
>>> I believe that many in the neuroscience community will be interested in a new paper that sheds light on why modularity evolves in biological networks, including neural networks. The same discovery also provides AI researchers a simple technique for evolving neural networks that are modular and have increased evolvability, meaning that they adapt faster to new environments.
>>> 
>>> Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. http://dx.doi.org/10.1098/rspb.2012.2863 (pdf)
>>> 
>>> Abstract: A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments). A key driver of evolvability is the widespread modularity of biological networks—their organization as functional, sparsely connected subunits—but there is no consensus regarding why modularity itself evolved. Although most hypotheses assume indirect selection for evolvability, here we demonstrate that the ubiquitous, direct selection pressure to reduce the cost of connections between network nodes causes the emergence of modular networks. Computational evolution experiments with selection pressures to maximize network performance and minimize connection costs yield networks that are significantly more modular and more evolvable than control experiments that only select for performance. These results will catalyse research in numerous disciplines, such as neuroscience and genetics, and enhance our ability to harness evolution for engineering pu!
>>> rposes.
>>> 
>>> Video: http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng
>>> 
>>> There has been some nice coverage of this work in the popular press, in case you are interested:
>>> 
>>> • National Geographic: http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/
>>> • MIT's Technology Review: http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/
>>> • Fast Company: http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning
>>> • Cornell Chronicle: http://www.news.cornell.edu/stories/Jan13/modNetwork.html
>>> • ScienceDaily: http://www.sciencedaily.com/releases/2013/01/130130082300.htm
>>> 
>>> I hope you enjoy the work. Please let me know if you have any questions.
>>> 
>>> Best regards,
>>> Jeff Clune
>>> 
>>> Assistant Professor
>>> Computer Science
>>> University of Wyoming
>>> jeffclune at uwyo.edu
>>> jeffclune.com
>>> 
>>> 
>> 
>> 
>> 
> 
> 
>