ontogenesis and synaptogenesis

[Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU]

Liz,

Thanks for posting that information on synaptogenesis.  It is extremely
valuable to have some knowledgeable people out there helping to filter and
interpret the neural-development literature for the rest of us.

I'm not sure to what extent your post was inspired by some of the recent
discussion of constructive/destructive algorithms (or "ontogenic", if you
prefer).  You're probably way ahead of me on this, but I just wanted to
mention that the obvious mapping of computational theories into
neuron-stuff isn't necessarily the only mapping or the best one.  For
example, some people automatically assume that what we call a "unit" must
be implemented as a neuron, and what we call a "weight" must be implemented
as an adjustable synapse.  But a "unit" might instead correspond to some
piece of the dendritic tree that happens to behave in a nonlinear way; on
the other hand, what we call a "unit" might be implemented as a whole
cluster of cooperating neurons.

The situation with constructive/destructive algorithms is similar.  The
obvious implementation of constructive algorithms would involve growing (or
recruiting) new neurons and adding them to the active network through the
creation of new synapses; destructive algorithms would presumably involve
elimination of existing synapses and neurons.  But that isn't the only
possible way of mapping these ideas to real neural systems.

For example, I usually describe the Cascade-Correlation architecture as
selecting new units out of a pool of candidates and "adding" them to the
active network.  But it is probably better to think of this event as a sort
of phase transition, which I jokingly call "tenure".  Before tenure, the
candidate units receive a full complement of inputs, and the input weights
are adjusted freely to maximize some measure of the candidate's potential
usefulness (currently, the degree of correlation with the remaining error).
Before tenure, these candidate units are either silent or their outputs are
ignored.  After tenure, the inputs are frozen (or at least much less
plastic), but now the rest of the net pays attention to that unit's output.
This can be a purely functional change; there doesn't really have to be any
visible change in the physical topology of the network.

It would be very interesting to know whether anything like this phase
transition actually occurs in individual neurons, but I have no idea how
one would go about looking for such a thing.

-- Scott Fahlman