What is a Symbol System?

[Dave.Touretzky@B.GP.CS.CMU.EDU]

Okay, I'll take a shot at responding to Stevan's query.  Of the eight
criteria he listed, I take exception to numbers 2 and 3, that rules must be
EXPLICIT and expressed as STRINGS of tokens.

In my recent work on phonology with Deirdre Wheeler (see "A connectionist
implementation of cognitive phonology", tech report number CMU-CS-89-144),
we define an architecture for manipulating sequences of phonemes.  This
architecture supports a small number of primitive operations like
insertion, mutation, and deletion of phonemes.  I claim that the rules for
deciding WHEN to apply these primitives do not need to be represented
explicitly or have a symbolic representation, in order for us to have a
symbol system.  It suffices that the rules' actions be combinations of the
primitive actions our architecture provides.  This is what distinguishes
our phonological model from the Rumelhart and McClelland verb learning
model.  In their model, rules have no explicit representation, but in
addition, rules operate directly on the phoneme sequence in a totally
unconstrained way, mapping activation patterns to activation patterns;
there are no primitive symbolic operations.  Therefore their model is
non-symbolic, as they themselves point out.

================

I also think the definition of symbol system Stevan describes is likely to
prove so constrained that it rules out human cognition.  This sort of
symbol system appears to operate only by disassembling and recombining
discrete structures according to explicit axioms.  What about more
implicit, continuous kinds of computation, like using spreading activation
to access semantically related concepts in a net?  How far the activation
spreads depends on a number of things, like the branching factor of the
semantic net, the weights on the links, and the amount of cognitive
resources (what Charniak calls "zorch") available at the moment.  (People
reason differently when trying to do other things at the same time, as
opposed to when they're relaxed and able to concentrate on a single task.)

Of course, spreading activation can be SIMULATED on a symbol processing
system, such as a Turing machine or digital computer, but this raises the
very important issue of levels of representation.  What the Physical Symbol
System Hypothesis requires is that the primitive atomic symbol tokens have
meaning IN THE DOMAIN of discourse we're modeling.  Although a Turing
machine can be made to simulate continuous computations at any level of
precision desired, it can only do so by using its primitive atomic symbols
in ways that have nothing to do with the semantic net it's trying to
simulate.  Instead its symbols are used to represent things like the
individual bits in some floating point number.  To play the Physical Symbol
Systems game correctly in the semantic net case, you have to choose
primitives corresponding to nodes and links.  But in that case there
doesn't seem to be room for continuous, non-compositional sorts of
computations.

Another problem I see with this definition of symbol system is that it
doesn't say what it means in #5 to "rulefully combine" symbols.  What about
stochastic systems, like Boltzmann machines?  They don't follow
deterministic rules, but they do obey statistical ones.  What about a
multilayer perceptron, which could be described as one giant rule for
mapping input patterns to output patterns?

-- Dave