Mathematical Tractability of Neural Nets

[AMR@IBM.COM]

I would like to commend the content though not the tone of
the recent remarks by Bates and Bower.  While all their
criticisms are certainly right, I would nevertheless hope
for more, not less, in the way of attempts (no matter how
quixotic) to reach across the gap that separates the disciplines
more than it does their respective subject matters.

It may be that biologists and linguists (to take two
extreme cases) will simply have to work each in their own
garden for a century, say, before their results on brain
activity begin to converge, and that current attempts at
a linkage of the kind that some connectionist literature
seems to envisage between neural architecture and cognitive
behavior are doomed to failure.

On the other hand, it seems to me that the history of science
is so full of cases where progress was delayed because people
were not ready to see connections between initially distinct
areas of study that the only lesson to draw is that if somebody
is willing to invest the time and the effort to pursue the connections,
let them.


I myself, as primarily a linguist, find myself in the following
predicament.  On the one hand, no one has shown me how linguistic
behavior can arise out of neural nets, but, on the other hand,
no one has shown me how it can arise out of formal grammars or
Turing machines, either.  I am very uncomfortable with the idea
that, given the obvious primitiveness of current neural models,
we should accept, as our idea of what makes language possible,
anything like a grammar in the conventional sense.  Linguists
of an earlier age were very careful (and most non-theoretically
inclined linguists still are) in viewing grammars as descriptions
of the data, not as models of anything that goes on inside
human beings.  Unfortunately, the direction of theoretical linguistics
has been largely one where the wonderful idea that we SHOULD
find out what goes on inside has been confused with the deplorable
conceit that we CAN find this out by doing little more than
armchair grammar writing but ATTRIBUTING to these grammars what
has come to be known as psychological reality.

A few of us are trying, in linguistics, to right the balance and
to develop an alternative in which the need to have grammars as
human-readable descriptions of masses of data does not lead to
the assumption that these same grammars are at all useful as
models of human cognition or neurology.  It may well be that
we will find that very few linguistic facts can be
explained, given current knowledge, by reference to something
more basic, and that the bulk of the routine grammatical
information that we know perfectly how to DESCRIBE will continue
to elude a psychological (much less a neurological) account
for a long time.  But it is a case, I submit, of having a choice
of a very few pearls or a whole lot of swine.  And in the search
for the former I am willing to get help anywhere I can find it.


Alexis Manaster-Ramer
IBM Research
POB 704
Yorktown Heights, NY 10510
(914) 784-7239