No subject

[Robin Fredericksen]

I apologize if I came across too sharply.  It's true that there needs to
be some well thought out terminological classification.  My response
in part is because I *AM* interested in the ontogeny of biological systems
and what it can teach us.  My personal research interests includes how
the biological system does what it does and how we can use that to our
advantage.  It is true that there are many researchers (as well as engineers)
who are looking at neural networks as a new and useful tool and could care
less how that tool relates to the systems that originally inspired the
technique (or at least invented it first :-) ).  We must find some way
to differentiate between models of (CNS) ontogeny for the study of (CNS)
ontogeny, and the use of ontogenically derived methods for finding the
correct network (CNS) structure to solve some specific computational
problem.  Yes, the word ontogeny is too general:

on.tog.e.ny \a:n-'ta:j-*-ne-\ n [ISV] : the development or course of
   development of an individual organism

(don't you just love the on-line Websters?)

We need to find a term that refers to *CNS* development.  *But*, I think
that use of the words 'destructive' and 'constructive' can
be just as misleading.  If I understand correctly (I was not at NIPS),
'destructive' is used to describe a network that begins with surfeit of
nodes and removes nodes during learning.  'Constructive' is then used
to describe networks that add nodes during learning.  A more accurate phrase
might be 'structurally self modifying' (or must we find a single word?  should
we then say it in German? :-) )  Are there such strongly qualitative
differences in networks resulting from the use of either strictly
'destructive' or 'constructive' rules that we need separate terminology?
We have available lots of verbs: alter, modify, transform, mutate, change,
vary (ad infinitem, almost; I don't have an on-line thesaurus).

tmb at ai.mit.edu says:
> ... Ontogeny is a very specializec (sic)
>process with a number of strict biological constraints operating; it is
>far from clear that processes that operate during ontogeny are similar
>or analogous to unit creation/destruction in an artificial neural
>network that learns.
>
>The thought of equating a mathematical and engineering technique with
>this biological process makes me cringe. The judgement is still out to
>what degree artificial "neural networks" are related to the real thing,
>so lets not aggravate the problem by introducing more biological
>terminology into the field.

Again, I agree that ontogeny is not the correct word, and I apologize for
defending it without suggesting a better one.  But is it not true that we
put lots of constraints on neural networks in order to coerce them into
'learning' the correct answer?  It may not be clear that there is any
similarity between the learning rules that use creation/destruction
of units in a network and 'rules' in the process of CNS development
(not ontogeny, which is too general a term), but I would also object
to having an artificial (ideological) wall put up between the concepts.

In the end, the real problem in selecting terminology lies in whether
you view a nn as an engineering tool that should be designed in a void,
completely derived without hints or clues from nature, or whether you
view it as a tool modeled after organisms (many of which have been
very successful in terms of evolution) from which we might learn.  My
personal bias is the latter.

INS_ATGE%jhuvms.hcf.jhu.edu at BITNET.CC.CMU.EDU writes:
>The other problem is that there might be learning in brain which involves
>recruitment of neurons or exclusion of neurons from the network performing a
>cognitive function (i.e. "software routing") which occurs much later than
>ontogeny (I do not know of results which show this, but constructive/
>destructive learning is so useful that it would seem useful if it was
>performed by "soft" changes in neural nets as opposed to actual
>synaptogenesis/cell death)(Do you know of any?).
>Anyway, I understand your point about reinventing wheels, but it seems
>that "ontogenic nets" seems too limiting to apply to "architecturally
>changeable" learning methods, with the exception of physiological
>learning which occurs during ontogeny.

First, Dale Purves feels that synaptogenesis continues throughout
the organisms life span (and thus CNS ontogeny).  Cell death occurs over
a much more limited space of time that depends on the complexity (size) of
the animal but is generally restricted to pre and a small postnatal period.
(Small in comparison with the animal's expected lifespan.) By software routing,
do you mean adding connections to new units?  That is just synaptogenesis and
its counterpart (synaptic loss).

Ok, so now I have wandered far away from defending the use of 'ontogenic'
and delved into my true reaction.  I was actually responding to what appeared
to be sarcasm (sorry Scott) but was intended to be humor.  My real objection
is to the building of walls within what should be an interdisciplinary
field through the use of imprecise or restricted terminology.  The problem
then is deciding what is precise, and that can come down to your personal view
of what nn research is for:  building tools, understanding biological
systems, or both at the same time.

So, any good ideas on terminology that fits the bill?

[More attempted humor...]
I wonder if constructive and destructive neural network learning rules will,
in the course of research, recapitulate the phylogeny of biological neural
networks?  We could separate out (on different continents, of course)
groups of researchers trying to develop a nn to solve the same class of
problems, say vision (email not allowed, of course).  Then after a suitable
amount of time, perhaps one group will have evolved rules for building
crustacean visual systems, another group will have evolved rules for the
building of arthropod visual systems, and yet another will be able to
build mammalian visual systems? But, will it take us hundreds of millions
of years? :-)

Eric Fredericksen