Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Juergen Schmidhuber]

The paper mentions that Santiago Ramón y Cajal already pointed out  
that evolution has created mostly short connections in animal brains.

Minimization of connection costs should also encourage modularization,  
e.g., http://arxiv.org/abs/1210.0118 (2012).

But who first had such a wire length term in an objective function to  
be minimized by evolutionary computation or other machine learning  
methods?
I am aware of pioneering work by Legenstein and Maass:

R. A. Legenstein and W. Maass. Neural circuits for pattern recognition  
with small total wire length. Theoretical Computer Science,  
287:239-249, 2002.
R. A. Legenstein and W. Maass. Wire length as a circuit complexity  
measure. Journal of Computer and System Sciences, 70:53-72, 2005.

Is there any earlier relevant work? Pointers will be appreciated.

Jürgen Schmidhuber
http://www.idsia.ch/~juergen/whatsnew.html




On Feb 10, 2013, at 3:14 AM, Jeff Clune wrote:

> Hello all,
>
> I believe that many in the neuroscience community will be interested  
> in a new paper that sheds light on why modularity evolves in  
> biological networks, including neural networks. The same discovery  
> also provides AI researchers a simple technique for evolving neural  
> networks that are modular and have increased evolvability, meaning  
> that they adapt faster to new environments.
>
> Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins  
> of modularity. Proceedings of the Royal Society B. 280: 20122863. http://dx.doi.org/10.1098/rspb.2012.2863 
>  (pdf)
>
> Abstract: A central biological question is how natural organisms are  
> so evolvable (capable of quickly adapting to new environments). A  
> key driver of evolvability is the widespread modularity of  
> biological networks—their organization as functional, sparsely  
> connected subunits—but there is no consensus regarding why  
> modularity itself evolved. Although most hypotheses assume indirect  
> selection for evolvability, here we demonstrate that the ubiquitous,  
> direct selection pressure to reduce the cost of connections between  
> network nodes causes the emergence of modular networks.  
> Computational evolution experiments with selection pressures to  
> maximize network performance and minimize connection costs yield  
> networks that are significantly more modular and more evolvable than  
> control experiments that only select for performance. These results  
> will catalyse research in numerous disciplines, such as neuroscience  
> and genetics, and enhance our ability to harness evolution for  
> engineering pu!
> rposes.
>
> Video: http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng
>
> There has been some nice coverage of this work in the popular press,  
> in case you are interested:
>
> • National Geographic: http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/
> • MIT's Technology Review: http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/
> • Fast Company: http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning
> • Cornell Chronicle: http://www.news.cornell.edu/stories/Jan13/modNetwork.html
> • ScienceDaily: http://www.sciencedaily.com/releases/2013/01/130130082300.htm
>
> I hope you enjoy the work. Please let me know if you have any  
> questions.
>
> Best regards,
> Jeff Clune
>
> Assistant Professor
> Computer Science
> University of Wyoming
> jeffclune at uwyo.edu
> jeffclune.com
>
>