Comments on Pinker's Replies to Harnad

[James L. McClelland]

I tried to send this yesterday but for some reason it
appears to have slipped through the cracks.  The copy I sent
to steve got through, but apparently the one to connectionists
didn't.
=============================================================
Steve --

In the first of your two messages,
there seemed to be a failure to entertain the possibility
that there might be a network that is not a strict implementation
of a rule system nor a pattern associator of the type
described by Rumelhart and me that could capture the
past tense phenomena.  The principle shortcoming of
our network, in my view, was that it treated the problem
of past-tense formation as a problem in which one generates
the past tense of a word from its present tense.  This
of course cannot be the right way to do things, for reasons
which you describe at some length in your paper.  However,
THIS problem has nothing to do with whether a network or
some other method is used for going from present to past tense.
Several researchers are now exploring models
that take as input a distributed representation
of the intended meaning, and generate as output
a description of the phonological properties of
the utterance that expresses that meaning.
Such a network must have at least one hidden layer to
do this task.  Note that such a network would
naturally be able to exploit the common structure
of the various different versions of English
inflectional morphology.  It is already clear that it
would have a much easier time learning inflection rather
than word-reversal as a way of mastering past tense
etc.

What remain to be addressed are issues about the
nature and onset of use of the regular inflection in
English.   Suffice it to say here that the claims you
and Prince make about the sharp distinction between
the regular and irregular systems deserve very close
scrutiny.  I for one find the arguments you give in
favor of this view unconvincing.

We will be writing at more length on these matters,
but for now I just wanted two points to be clear:

1) The argument about what class of models a
particular model's shortcomings exemplify is
not an easy one to resolve, and there is considerable
science and (yes) mathematics to be done to understand
just what the classes are and what can be taken as
examples of them.  Just what generalization you
believe you have reason to claim your arguments allow
you to make has not always been clear.  In the
first of your two recent messages you state:

  Our concern is not with (the mathematical question of) what
  nets can or cannot do in principle, but with which theories
  are true, and our conclusions were about pattern associators
  using distributed phonological representations.  We showed that
  it is unlikely that human children learn the regular rule
  the way such a pattern associator learns the regular rule,
  because it is simply the wrong tool for the job.

After receiving the message containing the above I wrote
the following:
<
 Now, the model Rumelhart and I proposed was a pattern associator
 using distributed phonological representations, but so are the
 other kinds of models that people are currently exploring; they
 happen though to use such representations at the output and not
 the input and to have hidden layers.  I strongly suspect that you
 would like your argument to apply to the broad class of models
 which might be encompassed by the phrase "pattern associators
 using distributed phonological representations", and I know for
 a fact that many readers think that this is what you
 intend.  However, I think it is much more likely that
 your arguments apply to the much narrower class of models
 which map distributed phonological representations of present
 tense to distributed phonological represenations of past tense.
>
In your longer, second note, you are very clear in stating
that you indend your arguments to be taken against the narrow
class of models that map phonology to phonology.
I do hope that this sensible view gets propagated, as
I think many may feel that you think you have a more general
case.  Indeed, your second message takes a general
attitude that I find I can agree with:

Let's do some more research and find out what
can and can't be done and what the important taxonomic classes
of architecture types might be.

2) There's quite a bit more empirical research to be done
even characterizing accurately the facts about the past
tense.  I believe this research will show that you have
substantially overstated the empirical situation in several
respects.  Just as one example, you and Prince state
the following:

   The baseball term _to fly out_, meaning 'make an out
   by hitting a fly ball that gets caught', is derived
   from the baseball noun _fly (ball)_, meaning 'ball hit
   on a conspicuously parabolic trajectory', which is in
   turn related to the simple strong verb _fly_, 'proceed
   through the air.  Everyone says 'he flied out';
   no mere mortal has yet been observed to have
   "flown out" to left field.

You repeated this at Cog Sci two weeks ago.  Yet in
October of 87 I received the message appended below,
which directly contradicts your claim.

As you state in your second, more constructive
message, we ALL need to be very clear about what the facts
are and not to rush around making glib statements!

        Jay McClelland
=======================================================
[The following is appended with the consent of the author.]

   Date: Sun, 11 Oct 87 21:20:55 PDT
   From: elman at amos.ling.ucsd.edu (Jeff Elman)
   To: der at psych.stanford.edu, jlm at andrew.cmu.edu
   Subject: flying baseball players

   Heard in Thursday's play-off game between the
   Tigers and Twins:

        "...and he flew out to left field...he's...OUT!"

   What was that P&P were saying?!

   Jeff

=======================================================