Systematicity & Fallacies: Boden & Niklasson

[Bob Hadley]

 The Fallacy of Equivocation:  Boden and Niklasson.


In a fairly recent paper (Connection Science, Vol. 12, 2000), 
Boden and Niklasson purport to  demonstrate that a collection 
of connectionist networks (call them c-nets) can display an important 
type of Strong Semantic Systematicity.
They make frequent references to my 1994 definitions of semantic systematicity
and to my papers on this important topic.   They also acknowledge that
in 1994 I published definite reservations about
claims by Niklasson and van Gelder to have produced a connectionist system 
that displays strong systematicity.

In their recent (2000) paper, Boden and Niklasson purport to have answered
my reservations by producing a case where a "novel test sentence" is 
assigned an appropriate meaning representation by  previously trained c-nets.
Readers may recall that my 1994 definition of strong semantic systematicity
required that the "previously trained c-net" must assign an appropriate
(and correct) meaning representation to a novel test sentence which contains
PREVIOUSLY KNOWN words in at least one novel position.  In contrast to this
requirement, the putative novel test sentence that Boden and Niklasson
employ does not present any previously known words in a novel position.
Rather, it presents a purportedly novel word in a known position.

However, there is a much more serious problem with their "novel test sentence"
(call this sentence S).  Here's the problem:  The supposed novel sentence S
does not produce a correct response when it is first presented to the
trained c-net.  So, Boden and Niklasson proceed to TRAIN the c-net on the
sentence S for an additional 1000 epochs (over and above the earlier training
phase).   In this latter training phase, only S is presented as input,
and backpropagation is employed.   Once this further training is complete,
Boden and Niklasson contend that a "novel" word in S has now been assigned
a meaning representation which they believe to be correct.  

But, of course, S is no longer a "novel test sentence" at this stage.
The c-net has been subjected to intensive training upon S, and only after
this further training is complete are Boden and Niklasson able to claim 
success.   Given this, for Boden and Niklasson to describe S as a novel
test sentence is (to express the matter diplomatically) to committ a 
serious instance of the fallacy of equivocation.  Indeed, I find it 
difficult to believe that Boden and Niklasson could be unaware that, as
most connectionists use the phrase "test data" (or "novel test sentence"),
sentence S is NOT a novel test sentence at all.  For this reason, it
astonishes me that Boden and Niklasson claim that they have NOW produced
an experimental result that satisfactorily answers my 1994 reservations
about the results published by Niklasson and van Gelder.

My 1994 reservations involved my  1994 definition of strong systematicity,
and that definition employed "novel test sentence" in the sense 
that connectionists commonly employ.  At best, Boden and Niklasson are 
assigning some new, and surprising sense to that phrase  -- hence the
fallacy of equivocation.

I believe there are other serious problems with Boden and Niklasson's 
(2000) paper, and I am presently writing a detailed critique of that 
paper.  I'll make my new paper available on the internet within a few weeks.
Look for a notice of my new critique on "Connectionist List" or send me an
email request for the pdf file.  

  
							   In astonishment,

							   Bob Hadley

Reference:  Boden, M. and Niklasson, L. (2000)
			"Semantic Systematicity and Context in Connectionist Networks",
       		 Connection Science, Vol. 12(2), pp. 111-142.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Robert F. Hadley (Bob)              Phone: 604-291-4488
  Professor                           email: hadley at cs.sfu.ca
  School of Computing Science         
  and Cognitive Science Program

  Simon Fraser University
  Burnaby, B.C. V5A 1S6
  Canada

  Web page:   www.cs.sfu.ca/~hadley/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~