Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Andrew Coward]

 In a paper published in 2001, I demonstrated that natural selection pressures on the brain resulting from: i. the need to learn with minimized interference with prior learning; ii. the need to "construct" the brain from DNA "blueprints" by a process that minimized the risk of errors; and iii. the need to minimize requirements for biological resources, led to some striking constraints on brain architectural forms.

One of the constraints identified in the paper is the need for a modular hierarchy in which modules are made up of submodules, submodules of sub-sub-modules and so on, in such a way that connectivity is minimized, and there is much less connectivity between modules than within their submodules and so on. I pointed out that the cortex has a striking resemblance to this type of hierarchy.

I have subsequently applied these ideas to developing ways to use the architectural forms to understand cognition in terms of anatomy physiology, and I have been teaching a course on this topic for a number of years.

The 2001 paper is Coward, L.A. (2001). The Recommendation Architecture:
lessons from the design of large scale electronic systems for cognitive
science. Journal of Cognitive Systems Research 2(2), 111-156. It can be downloaded from my website along with more recent papers.

Andrew Coward
http://cs.anu.edu.au/~Andrew.Coward/index.html
 

On 22/02/13, Juergen Schmidhuber  <juergen at idsia.ch> wrote:
> The paper mentions that Santiago Ramón y Cajal already pointed out that evolution has created mostly short connections in animal brains.
> 
> Minimization of connection costs should also encourage modularization, e.g., http://arxiv.org/abs/1210.0118 (2012).
> 
> But who first had such a wire length term in an objective function to be minimized by evolutionary computation or other machine learning methods?
> I am aware of pioneering work by Legenstein and Maass:
> 
> R. A. Legenstein and W. Maass. Neural circuits for pattern recognition with small total wire length. Theoretical Computer Science, 287:239-249, 2002.
> R. A. Legenstein and W. Maass. Wire length as a circuit complexity measure. Journal of Computer and System Sciences, 70:53-72, 2005.
> 
> Is there any earlier relevant work? Pointers will be appreciated.
> 
> Jürgen Schmidhuber
> http://www.idsia.ch/~juergen/whatsnew.html
> 
> 
> 
> 
> On Feb 10, 2013, at 3:14 AM, Jeff Clune wrote:
> 
> >Hello all,
> >
> >I believe that many in the neuroscience community will be interested in a new paper that sheds light on why modularity evolves in biological networks, including neural networks. The same discovery also provides AI researchers a simple technique for evolving neural networks that are modular and have increased evolvability, meaning that they adapt faster to new environments.
> >
> >Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. http://dx.doi.org/10.1098/rspb.2012.2863 (pdf)
> >
> >Abstract: A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments). A key driver of evolvability is the widespread modularity of biological networks—their organization as functional, sparsely connected subunits—but there is no consensus regarding why modularity itself evolved. Although most hypotheses assume indirect selection for evolvability, here we demonstrate that the ubiquitous, direct selection pressure to reduce the cost of connections between network nodes causes the emergence of modular networks. Computational evolution experiments with selection pressures to maximize network performance and minimize connection costs yield networks that are significantly more modular and more evolvable than control experiments that only select for performance. These results will catalyse research in numerous disciplines, such as neuroscience and genetics, and enhance our ability to harness evolution for engineering pu!
> >rposes.
> >
> >Video: http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng
> >
> >There has been some nice coverage of this work in the popular press, in case you are interested:
> >
> >• National Geographic: http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/
> >• MIT's Technology Review: http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/
> >• Fast Company: http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning
> >• Cornell Chronicle: http://www.news.cornell.edu/stories/Jan13/modNetwork.html
> >• ScienceDaily: http://www.sciencedaily.com/releases/2013/01/130130082300.htm
> >
> >I hope you enjoy the work. Please let me know if you have any questions.
> >
> >Best regards,
> >Jeff Clune
> >
> >Assistant Professor
> >Computer Science
> >University of Wyoming
> >jeffclune at uwyo.edu
> >jeffclune.com
> >
> >
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20130222/1776d3a1/attachment.html>