Connectionists: New paper on why modules evolve, and how to evolve modular neural networks

[A.S.]

Dear colleagues,
When we look for reasons why modules are formed, we have assumed from 
the outset that modularity is an ubiquitous property of the brain 
(neural systems) as a cognitive system.  Next the localization principle 
often comes into play, assuming an one-to-one relationship between the 
corresponding structural and functional modules. The hypothesis of the 
columnar organization of the cerebral cortex basically rests on this 
idea: columns are structural modules computing certain basis functions, 
and the columnar network computes any reasonable (cognitive) function 
(e.g. Maas and Markram´s 2006  model of how cortical microcircuits 
compute cognitive functions).

Along these lines of thinking the method of reverse engineering works:

1. Capacity analysis: Specify a certain cognitive capacity which is 
assumed to be produced through the cortex by computing a certain function.

2. Decompositional analysis:
(a) Functional (computational) analysis: Select a set of basis functions 
which might serve as functional components or computational units in the 
cortex.
(b) Structural analysis: Identify a set of anatomical components of the 
cortex. Provide evidence that cortical microcircuits are the anatomical 
components of the cortex.

3. Localization: Provide evidence for the functional / computational 
components being linked with the anatomical components.

4. Synthesis:
(a) Modeling:
i. Establish a structurally adequate functional model of the 
computational unit (the presumed 'canonical circuit') which computes the 
basis functions of step 2.(a).
ii. Build a structurally adequate network model of the cortex (or some 
subsystem) composed of the canonical circuit models.
(b) Simulation: Prove that the specific cognitive capacity or function 
under study is computed by the network of circuit models, i.e. through 
superposition of the specified basis functions.

This `recipe´ - if reasonable - would be fine. We know, however, there 
are serious problems with this method. In short:

@ 1. Specification of a cognitive capacity:
Requires a taxonomy of cognitive processes which is out of sight, as is 
obvious from recent attempts to build cognitive ontologies.
@ 2.-3. Decomposition -Localization:
It has been impossible to find the cortical microcircuit that computes a 
specific basis function. No genetic mechanism has been deciphered that 
designates how to construct a column. The column structures encountered 
in many species (but not in all) seem to represent spandrels.
@ 4. Synthesis / Proof by simulation:
Sure, producing and understanding complex phenomena from the interaction 
of simple nonlinear elements like artificial neurons or cellular 
automata is possible. One expects then, that this would also work for 
cortical circuits which are recognized as nonlinear devices, and 
theories could be applied (or developed, if not yet available) that 
would guide us to which model setup might have generated a given network 
behavior.
However, inverse problems in complex systems  (which processes caused a 
specific complex behavior of a given system?) are hard because of  
ill-posedness. Thus,  from observed activity or function  of cortical 
circuits and networks we cannot, in principle, infer the internal 
organization, and the proof is not possible that the particular 
cognitive capacity under study is generated by the network model.

My conclusion is: In cognitive / computational neuroscience we (should) 
deal with complex, integrated systems. This means, there is no "natural" 
way to decompose or modularize the brain, neither structurally nor 
functionally!

Details of the arguments can be found here:

Schierwagen, A.: On Reverse Engineering in the Cognitive and Brain 
Sciences.  Natural Computing:  11 (2012), 141-150, 
doi:10.1007/s11047-012-9306-0 
<http://www.informatik.uni-leipzig.de/%7Eschierwa/2012_Schierwagen_Rev_Engn.pdf>


Best wishes,

Andreas
---------------------------------------------------------------
Prof. Dr. Andreas Schierwagen
Universität Leipzig, Institut für Informatik, Germany
http://www.informatik.uni-leipzig.de/~schierwa/ 
<http://www.informatik.uni-leipzig.de/%7Eschierwa/>



Am 24.02.2013 16:05, schrieb Tony Prescott:
> Dear colleagues,
>
> The Clune et al. article we are discussing mentions that selection for
> reduced connectivity could be a "spandrel" (the consequence of
> selection for something else) but does not explore this possibility in
> much depth.  In the case of biological brains it is hard to see why
> low connectivity should be directly selected rather than arising
> through the need to keep a lid on the size and metabolic cost of
> maintaining the brain.  A 1991 paper by Ringo
> (http://www.ncbi.nlm.nih.gov/pubmed/1657274) shows that larger brains
> cannot maintain the same degree of inter-connectedness as smaller ones
> and therefore long-range connections are necessary sparser if
> increased an in neuron count is not going to give rise to an
> exponential increase in brain size.  Reduced connectivity is therefore
> an architectural constraint for larger brains in not too dissimilar
> way to the need for spandrels in cathedral domes (as discussed by
> Gould, 1979).
>
> An important consideration for biological brains is connection length.
>   Leise 1990 (http://www.ncbi.nlm.nih.gov/pubmed/2194614) provides a
> useful summary of the reasons why nervous systems are composed of
> physically modular components with a high number of short-range
> connections and low number of longer range ones.  As the literature on
> small world networks show, however, it is important not to assume that
> physical modularity requires functional modularity.  Appropriate
> sparse connectivity can allow fast communication and synchronisation
> across large  networks that can support distributed functional
> modules.
>
> Regards,
>
> Tony Prescott
>
>
>
> On 24 February 2013 03:49, Terry Sejnowski <terry at salk.edu> wrote:
>> G. Mitchison, Neuronal branching patterns and the economy of cortical wiring, Proc. Roy. Soc. London
>> B Biol. Sci. 245 (1991) 151{158
>>
>> D.B. Chklovskii, C.F. Stevens, Wiring optimization in the brain, Neural Information Processing Systems
>> (1999)
>>
>> Koulakov AA, Chklovskii DB. Orientation preference patterns in mammalian visual cortex: a wire length minimization approach.  Neuron. 2001 Feb;29(2):519-27.
>>
>> Chklovskii DB, Schikorski T, Stevens CF. Wiring optimization in cortical circuits.
>> Neuron. 2002 Apr 25;34(3):341-7.
>>
>> Terry
>>
>> -----
>>
>>> The paper mentions that Santiago Ram<F3>n y Cajal already pointed out
>>> that evolution has created mostly short connections in animal brains.
>>>
>>> Minimization of connection costs should also encourage modularization,
>>> e.g., http://arxiv.org/abs/1210.0118 (2012).
>>>
>>> But who first had such a wire length term in an objective function to
>>> be minimized by evolutionary computation or other machine learning
>>> methods?
>>> I am aware of pioneering work by Legenstein and Maass:
>>>
>>> R. A. Legenstein and W. Maass. Neural circuits for pattern recognition
>>> with small total wire length. Theoretical Computer Science,
>>> 287:239-249, 2002.
>>> R. A. Legenstein and W. Maass. Wire length as a circuit complexity
>>> measure. Journal of Computer and System Sciences, 70:53-72, 2005.
>>>
>>> Is there any earlier relevant work? Pointers will be appreciated.
>>>
>>> Juergen Schmidhuber
>>> http://www.idsia.ch/~juergen/whatsnew.html
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20130226/d9c512a5/attachment-0001.html>