Connectionists: modularity vs. sparsity

[Juyang Weng]

You wrote:  "it's not surprising that they got modularity because they 
placed a cost on connection length."

If modularity is primarily due to connection length minimization, this 
explanation puts most factors into evolution.  This should be largely 
wrong.  Why?  You consider only computer simulations but not biology.   
Biology seems to be smarter.

According to my understanding of brain plasticity, the connections in 
the brain are primarily due to prenatal and postnatal activities during 
development. That is, signals from firing neurons guide other neurons to 
wire, not primarily genes!

By view: the brain modularity is primarily due to
(A) sensor discontinuities (e.g., between eye and skin) and continuities 
(e.g., between neighboring receptors on the retina of sensors), and
(B) effector discontinuities (e.g., between hand muscles and leg 
muscles) and continuities (e.g., between neighboring muscles in the same 
figure)

Note: not just sensors, but also effectors. According to what I 
understand from the neuroanatomy and our brain-inspired DN 
(Developmental Network) model, the sources of signals that guide the 
brain wiring are not only sensors (receptors) but also effectors 
(muscles and glands).

I know that I am among a very small minority to hold this view, but I 
hope that more and more people can see that this is true.

The major drawback of many existing studies discussed here is that the 
researchers confuse evolution with development --- they have little idea 
what a call does during development.  I suggest that everybody learn 
neurobiology to understand how each neuron migrates and connects during 
development to form tissues (e.g., cortex).  The genes inside a cell 
determine largely what a cell is now, but how a cell wires is largely 
determined by the activities of other cells that this cell communicates 
using what is called cell-to-cell signaling.

All those who took BMI 811 Biology for Brain-Mind Research last summer 
have learned the above.  I encourage that you take advantage of
BMI summer school 2013, which will be announced soon.   Please let me 
know if you are interested in but have some blackout dates, so that we 
can do our best to avoid your blackout dates.  A BMI course will last 
for three weeks in this summer.

Now, you can take a look at the BMI program last year:
http://www.brain-mind-institute.org/programs.html


-John


On 3/4/13 11:53 AM, Joshua C. Bongard wrote:
> A lot of discussion here lately about the evolution of modularity,
> sparked by the recent paper
>
> http://arxiv.org/abs/1207.2743
>
> In it, the authors evolve ANNs to perform a specific task while placing
> a penalty on total wiring length.
>
> There were a lot of responses here along the lines of "it's not 
> surprising
> that they got modularity because they placed a cost on connection 
> length."
>
> However, what I found interesting about this paper is that they got
> modularity rather than sparsity. It is important to keep in mind that
> these two properties of networks are not the same thing.
>
> So, if I were to use Clune et al.'s approach to evolve ANNs for another
> task, would I get modularity or sparsity?
>
> Food for thought.
>
>

-- 
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
3115 Engineering Building
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: weng at cse.msu.edu
URL: http://www.cse.msu.edu/~weng/
----------------------------------------------