Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[Juyang Weng]

In terms of genetic algorithms (evolution), I guess that they are useful 
probably for fine tuning some species-specific global parameters.
However, the evolved biology (e.g., genome) seems to make sure that the 
structured (not modular) brain (coarse structure) is,
in the first order approximation, largely shaped by statistics of 
signals observed during postnatal developmental activities.
By activities, I do not mean just autonomous activities of the 
individual, since every cell sends signals to other cells
during the entire process of development.

In other words, the brain seems to be very different from other organs 
in the body if one thinks that evolution has largely determined
an organ (other than the brain).

-John


On 2/27/13 3:48 PM, Jaap Murre wrote:
>
> An approach I found was interesting but which has not been pursued 
> very much is a 'four-step' approach to modelling:
>
> 1. genetic algorithm -> 2. neural network architecture -> 3. exposure 
> to patterns in environment -> 4. test for generalization
>
> This corresponds very roughly to
>
> 1. evolution -> 2. structured (e.g., modular) brain (coarse structure) 
> -> 3. developmental phase (fine structure) -> 4. behavior
>
> When we tried this approach, we were surprised by the interesting 
> neural architectures we obtained in a number recognition task. The 
> evolved architectures showed better generalization than the ones we 
> had thought up ourselves and they showed better performance (faster 
> development and better generalization). Also, many of the efficient 
> architectures incorporated elements we find in the visual system, such 
> as coarse-grained versus fine-grained processing (cf. dorsolateral vs 
> ventral stream) and extensive recurrent connectivity whereby higher 
> areas feed back to lower areas.
>
> Happel, B. L. M., & Murre, J. M. J. (1994). Design and evolution of 
> modular neural-network architectures. Neural Networks, 7(6-7), 985-1004.
>
> We also did a mathematical analyses of the feasibility of modular 
> neural networks in terms of wiring length and brain volume, aiming to 
> incorporate as much as possible the quantitative neuroanatomy known in 
> the early nineties. We found that such architectures would indeed fit 
> into our skull, whereas say a sparse random architecture would not.
>
> Murre, J. M. J., & Sturdy, D. P. F. (1995). The connectivity of the 
> brain: Multi-level quantitative analysis. Biological Cybernetics, 73, 
> 529-545.
>
> A surprising outcome to us in this study was that when working from 
> first principles, in large brain structures (above a certain number of 
> neurons), having a architecture with a cortex and white matter 
> underneath is actually more efficient (less volume) than when white 
> and grey matter are mixed together. Intuitively, a cortex architecture 
> seems to waste space and increase wiring length but this does not seem 
> to be the case, which may explain why large brains use it so much.

-- 
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
3115 Engineering Building
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: weng at cse.msu.edu
URL: http://www.cse.msu.edu/~weng/
----------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20130227/fc7d000d/attachment.html>