Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Jeff Clune]

Dear John Weng,

Our paper on the evolutionary origins of modularity focuses on modularity in biological networks in general, not just neural networks. The dynamics you describe are not relevant to all biological networks (e.g. metabolic networks, genetic regulatory networks, protein-protein interaction networks, etc.). However, we are particularly interested in the evolution of neural modularity, and neural networks have more obvious connection costs than many biological networks, so we do think our paper sheds light on the origins of neural modularity as well. 

On that front, while I of course agree that each connection in a complex neural wiring diagrams is not genetically specified, genes ultimately encode the rules that govern neural development. Evolution may thus favor modularity via selection for certain types of developmental programs (e.g. those that tend to produce fewer, shorter connections). Just because development plays a substantial role does not mean that genes do not as well. One can easily imagine selection for developmental programs that lead to fully connected neural networks, but that did not occur in nature. A major force that prevented that from happening is likely the many different costs that would be incurred for all those connections (including the cranial space to house them). Our work suggests that minimizing connection costs leads to modularity; that minimization could be accomplished via genetically-encoded developmental rules. 

I think our results are thus complementary with work investigating how neural development is biased towards creating modular connectivity patterns, and may even suggest a reason why there was selection for such developmental biases in the first place. 


Best regards,
Jeff Clune

Assistant Professor
Computer Science
University of Wyoming
jeffclune at uwyo.edu
jeffclune.com

On Mar 30, 2013, at 1:21 PM, Juyang Weng <weng at cse.msu.edu> wrote:

> Dear Jeff Clune:
> 
> Thank you for pointing to the URL.  I quote some statements below in two paragraphs.   Although I agree that the genome has made a "best guess" when a zygote forms, it is simple-minded to attribute the modularity of the brain, even at the birth time, primarily to "evolution of modularity" as you put it.  In other words, unlike the zygote, the brain of a new born is no longer simply the "best guess" of the genome.  The body of the new born has played a fundamental role in the formation of the modularity inside the newborn's brain.   Namely, the "emergence" or development, is the key process for brain's modularity in the newborn and of course also in the later life.  
> If you have a chance to read our computational model of the DEVELOPMENT of a brain-inspired network DN, at least computationally DN does not need to attribute its emergence of modularity to anything other than a set of cell mechanisms.  This is because of the cell-centered role of the genes, known as genomic equivalence.  For example, each cell grows and connects according to signals from other cells in its neighborhood (not primarily genes!).  Many biological experiments have shown how autonomous cells (whose properties are
> to some degree genome specified) communicate to migrate, differentiate, form tissues (e.g., cortex), and connect.  In our DN model, such cell behaviors give rise to surprising brain-like capabilities when sensory and motor signals are present. 
> By attention to "emergence" in the paragraphs I quoted below. 
> -John 
> "The existence of modules is recognized at all levels of the biological hierarchy. In order to understand what modules are, why and how they emerge and how they change, it would be necessary to start a joint effort by researchers in different disciplines (evolutionary and developmental biology, comparative anatomy, physiology, neuro- and cognitive science). This is made difficult by disciplinary specialization. [...] we claim that, because of the strong similarities in the intellectual agenda of artificial life and evolutionary biology and of their common grounding in Darwinian evolutionary theory, a close interaction between the two fields could easily take place. Moreover, by considering that artificial neural networks draw an inspiration from neuro- and cognitive science, an artificial life approach to the problem could theoretically enlarge the field of investigation." (Calabretta et al., 1998)
> 
> A general definition of modularity and nonmodularity in neural networks can be the following: “modular systems can be defined as systems made up of structurally and/or functionally distinct parts. While non-modular systems are internally homogeneous, modular systems are segmented into modules, i.e., portions of a system having a structure and/or function different from the structure or function of other portions of the system. [...] In a nonmodular architecture one and the same connection weight may be involved in two or more tasks. In a modular architecture each weight is always involved in a single task: Modules are sets of ‘proprietary’ connections that are only used to accomplish a single task.” (Calabretta & Parisi, 2005, Fig. 14.4; see also Calabretta et al., 2003).
> 
> On 3/29/13 8:30 PM, Jeff Clune wrote:
>> Hello Christos,
>> 
>> Rafael Calabretta keeps a list of papers on the subject of the evolution of modularity. 
>> 
>> http://gral.ip.rm.cnr.it/rcalabretta/modularity.html
>> 
>> I like your idea of a wiki too. It could be a great resource for the field. We could even start fleshing out this page, which is currently nearly empty: http://en.wikipedia.org/wiki/Modularity_(biology)
>> 
>> PS. Thanks to everyone who has participated in the discussion of our paper The Evolutionary Origins of Modularity. Some of the papers that have been mentioned we reference in our paper, and others are new to us. We have enjoyed learning about the various different studies and opinions on this subject, and look forward to more great work to come. 
>> 
>> 
>> Best regards,
>> Jeff Clune
>> 
>> Assistant Professor
>> Computer Science
>> University of Wyoming
>> jeffclune at uwyo.edu
>> jeffclune.com
>> 
>> On Mar 29, 2013, at 3:31 PM, Christos Dimitrakakis <christos.dimitrakakis at gmail.com> wrote:
>> 
>>> Dear all,
>>> 
>>> Is there no survey or taxonomy that discusses this line of work in one 
>>> place?
>>> If not, I have a suggestion. Why not start up a wiki to begin with? That 
>>> would also be of tremendous aid to any newcomers.
>>> 
>>> Best,
>>> Christos
>>> 
>>> -- 
>>> Dr. Christos Dimitrakakis
>>> http://lia.epfl.ch/People/dimitrak/
>>> 
>> 
> 
> -- 
> --
> Juyang (John) Weng, Professor
> Department of Computer Science and Engineering
> MSU Cognitive Science Program and MSU Neuroscience Program
> 428 S Shaw Ln Rm 3115
> Michigan State University
> East Lansing, MI 48824 USA
> Tel: 517-353-4388
> Fax: 517-432-1061
> Email: weng at cse.msu.edu
> URL: http://www.cse.msu.edu/~weng/
> ----------------------------------------------
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20130401/c3c785fa/attachment.html>