Mathematical Tractability of Neural Nets

[Elizabeth Bates]

I think you need to distinguish between neuroscience in general (where
significant progress is being made in many areas), and the particular
area of neurology, with particular reference to language.  I think
that someday, in retrospect, we will see that progress WAS made in
the neurology of language during this period in our history, but much
of that progress will prove to be the debunking of classic disconnection
and localization theories.  Witness, for example, the stunning papers
by Posner, Pedersen, Fox, Raichle, etc. on metabolic activity during
language use -- fascinating, but only marginally compatible with
anything that we previously believed.  Looking at a PET scan or and
ERP study of "live" language use, one can only be impressed with
HOW MUCH of the brain is very active during language use -- which,
of course, fits with other anomalous findings that have been around
but largely ignored (e.g. Ojemann's findings on the many many different
points in the brain that can interrupt language processing when an
electric stimulus is applied during cortical mapping prior to surgery).
We are, without question, in a period of transition and serious
rethinking.  For example, Geoff Hinton and Tim Shallice (a former
believer in old-fashioned localization) have been carrying out
simulations in which a neural network is trained up on some language
task and then "lesioned".  Some very specific but totally unexpected
"syndromes" fall out of randomly placed or indeed randomly distributed
damage to the net.  Specific syndromes can be a "local minimum", a
fact about the mathematics of a distributed network rather than a
result (of the typical sort) induced by "subtracting" some local
and highly specific piece-of-the-machine.  

When you were trying to recommend "findings" by "neurologists" that
connectionists should follow, you stressed some classic claims about
Grammar (Broca's area), semantics (Wernicke's area), frontal lobs
(that's lobes -- speech initiation), in short the Geschwind view
that was so popular through the 1970's.  That is the view that I am
objecting to now, not the more general and indeed very fruitful
union between neuroscience and computation.  One cannot compare our
knowledge of the visual system (which is extensive) with our knowledge
of how the brain is organized for language (which is, right now, totally
up for grabs). -liz bates