Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Richard Loosemore]

I hate to say this, but during discussions with fellow students back in 
1987, I remember pointing out that it was not terribly surprising that 
the cortex consisted of columns (i.e. modules) with dense internal 
connectivity, with less-dense connections between columns -- not 
surprising, because the alternative was to try to make the brain less 
modular and connect every neuron in each column to all the neurons in 
all the other columns, and the result would be brains that were a 
million times larger than they are (due to all the extra wiring).

The same logic applies in all systems where it is costly to connect 
every element to every other: the optimal connectivity is 
well-connected, tightly clustered groups of elements.

During those discussions the point was considered so obvious that it 
sparked little comment. Ever since then I have told students in my 
lectures that this would be the evolutionary reason for cortical columns 
to exist.

So I am a little confused now. Can someone explain what I am missing 
.........?

Richard Loosemore
Department of Physical and Mathematical Sciences,
Wells College



On 2/13/13 9:48 AM, Juergen Schmidhuber wrote:
> The paper mentions that Santiago Ramón y Cajal already pointed out 
> that evolution has created mostly short connections in animal brains.
>
> Minimization of connection costs should also encourage modularization, 
> e.g., http://arxiv.org/abs/1210.0118 (2012).
>
> But who first had such a wire length term in an objective function to 
> be minimized by evolutionary computation or other machine learning 
> methods?
> I am aware of pioneering work by Legenstein and Maass:
>
> R. A. Legenstein and W. Maass. Neural circuits for pattern recognition 
> with small total wire length. Theoretical Computer Science, 
> 287:239-249, 2002.
> R. A. Legenstein and W. Maass. Wire length as a circuit complexity 
> measure. Journal of Computer and System Sciences, 70:53-72, 2005.
>
> Is there any earlier relevant work? Pointers will be appreciated.
>
> Jürgen Schmidhuber
> http://www.idsia.ch/~juergen/whatsnew.html
>
>
>
>
> On Feb 10, 2013, at 3:14 AM, Jeff Clune wrote:
>
>> Hello all,
>>
>> I believe that many in the neuroscience community will be interested 
>> in a new paper that sheds light on why modularity evolves in 
>> biological networks, including neural networks. The same discovery 
>> also provides AI researchers a simple technique for evolving neural 
>> networks that are modular and have increased evolvability, meaning 
>> that they adapt faster to new environments.
>>
>> Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins 
>> of modularity. Proceedings of the Royal Society B. 280: 20122863. 
>> http://dx.doi.org/10.1098/rspb.2012.2863 (pdf)
>>
>> Abstract: A central biological question is how natural organisms are 
>> so evolvable (capable of quickly adapting to new environments). A key 
>> driver of evolvability is the widespread modularity of biological 
>> networks—their organization as functional, sparsely connected 
>> subunits—but there is no consensus regarding why modularity itself 
>> evolved. Although most hypotheses assume indirect selection for 
>> evolvability, here we demonstrate that the ubiquitous, direct 
>> selection pressure to reduce the cost of connections between network 
>> nodes causes the emergence of modular networks. Computational 
>> evolution experiments with selection pressures to maximize network 
>> performance and minimize connection costs yield networks that are 
>> significantly more modular and more evolvable than control 
>> experiments that only select for performance. These results will 
>> catalyse research in numerous disciplines, such as neuroscience and 
>> genetics, and enhance our ability to harness evolution for 
>> engineering pu!
>> rposes.
>>
>> Video: 
>> http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng
>>
>> There has been some nice coverage of this work in the popular press, 
>> in case you are interested:
>>
>> • National Geographic: 
>> http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/
>> • MIT's Technology Review: 
>> http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/
>> • Fast Company: 
>> http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning
>> • Cornell Chronicle: 
>> http://www.news.cornell.edu/stories/Jan13/modNetwork.html
>> • ScienceDaily: 
>> http://www.sciencedaily.com/releases/2013/01/130130082300.htm
>>
>> I hope you enjoy the work. Please let me know if you have any questions.
>>
>> Best regards,
>> Jeff Clune
>>
>> Assistant Professor
>> Computer Science
>> University of Wyoming
>> jeffclune at uwyo.edu
>> jeffclune.com
>>
>>
>
>
>