Connectionist symbol processing: any progress?

[Lev Goldfarb]

On Thu, 13 Aug 1998, Neil W Rickert wrote:

> On Wed, 12 Aug 1998 Lev Goldfarb wrote:
> 
> >                                         Hacking apart, the INPUT SPACE of
> >a learning machine must be defined axiomatically, as is the now universal
> >practice in mathematics. These axioms define the BASIC OPERATIONAL BIAS of
> >the learning machine, i.e. the bias related to the class of permitted
> >object operations (compare with the central CS concept of abstract data
> >type).
> 
> Why?  This seems completely wrong to me.
> 
> It seems to me that what you have stated can be paraphrased as:
> 
>    The knowledge that the learning machine is to acquire must be
>    pre-encoded into the machine as (implicit or explicit) innate
>    structure.

Neil,
 
Your paraphrase is wrong: my statement refers to the fact that the
fundamental bias related to the (evolutionary) structure of the
environment must be postulated. 

[Again, compare with the concept of abstract data type (ADT), which has
been found to be ABSOLUTELY indispensable in computer science for dealing
with various type of data]. 

> But, if my paraphrase is correct, then one wonders why such a machine
> warrants the name "learning machine."
> 
> Presumably we have very different ideas as to what is learning.  I
> take it that human learning is the process of discovering the nature
> of our world.  More generally I take it that, at its most fundamental
> level, learning is the process of discovering the structure of the
> input space, a process which may require the eventual production of
> an axiomatization (a set of natural laws) for that space.

I am suggesting "that, at its most fundamental level, learning is the
process of discovering" the inductive class structure and not "the
structure of the input space". The latter would be simply impossible to
discover unless one is equipped with exactly the same bias. 

If the machine is not equipped with the "right" evolutionary bias (about
the COMPOSITIONAL STRUCTURE OF OBJECTS in the universe) hardly any
reliable inductive learning is possible. On the other hand, if it is
equipped with the right bias (about the overall structure of objects) then
the inductive learning could proceed in the biologically familiar manner. 

I believe that the "tragedy" of the connectionist symbol processing is
directly related to the fact that the vector space bias is structurally
too simple/restrictive and has hardly anything to do with the symbolic
bias, the latter being a simple example of the "right"  structural
evolutionary bias of the universe.

Cheers,
         Lev