What have neural networks achieved?

[Jay McClelland]

I haven't been able to read all of the email on connectionists lately
and so it is possible that the following is redundant, but it seems to
me there is a real success story here.

There has been a great deal of connectionist work on the processing of
regular and exceptional material, initiated by the
Rumelhart-McClelland paper on the past tense.  Debate has raged on the
subject of the past tense and work there is ongoing, but I won't claim
a success story there at this time.  What I would like to point to
instead is the related topic of single word reading.  Sejnowski and
Rosenberg's NETTALK first extended connectionist ideas to this issue,
and Seidenberg and McClelland went on to show that a connectionist
model could account in great detail for the pattern of reaction times
found in around 30 studies concerning the effects of regularity,
frequency, and lexical neighbors on reading words aloud.  This was
followed by a resounding critique along the lines of Pinker and
Prince's critique of R&M, coming this time from Derrick Besner (and
colleagues) and Max Coltheart (and colleagues).  Both pointed to the
fact that the S&M model didn't do a very good job of reading nonwords,
and both claimed that this reflected an in-principal limitation of a
connectionist, single mechanism account: To do a good job with both,
it was claimed, a dual route system was required.

The success story is a paper by Plaut, McClelland, Seidenberg, and
Patterson, in which it was shown in fact that a single mechanism,
connectionist model can indeed account for human performance in
reading both words and nonwords.  The model replicated all the S&M
findings, and at the same time was able to read non-words as well as
human subjects, showing the same types of neighbor-driven responses that
human readers show (eg MAVE is sometimes read to rhyme with HAVE
instead of SAVE).

Of course there are still some loose ends but it is no longer possible
to claim that a single-mechanism account cannot capture the basic
pattern of word and non-word reading data.  

The authors of PMSP all believe, I think, that there are semantic as
well as phonological sources of influence on word reading, so that the
system is, to an extent, a kind of dual-route system.  This was in
fact articulated in the earlier, SM formulation.  This can lead to
apparent dissociations in fMRI and effects of brain damage on reading,
but the dissociation is fundamentally one of semantic vs phonological
processes rather than lexical vs rule-guided processes.  For example
the phonological system, while sensitive to regularities, nevertheless
captures knowledge of specific high-frequency exceptions.

-- Jay McClelland