'natural' concepts (symbol grounding)

[Ross Gayler]

Peter Foldiak <pf103 at phx.cam.ac.uk> writes:

>[Learning concepts by pure observation] may not be easy,
>but I don't think it is generally impossible.
>Barlow's redundancy reduction principle, for instance, would say that
>features that result in lower statistical redundancy are better.
>(Redundancy here is not first-order (bit probabilities) but pairwise
>and higher-order redundancy.)

OK, let me expand on my position a little.  I realise that connectionist systems
can learn to categorise inputs and can generalise on this task.  I'm not
entirely convinced that this warrants being called 'concept learning' (but
I don't think I can justify this belief).  More importantly, most systems
have had a considearble amount of effort put into the architecture and 
training set to ensure that they categorise and generalise as expected.

My point is that models of the input data can be compared in terms of
redundancy - but having a measure of goodness of fit does not directly
help construct an optimal network.  If the input can be simply
characterised in terms of a higher order redundancy, it is very unlikely
that a net with no representational biases, starting from input at the
pixel level, will discover anything close to the optimal model inside
a pragmatically bounded time.  This is what I meant when I said that
learning from observation was generally impossible.

The point I wanted to make was that I think learning at a pragmatically
useful rate in a complex environment requires the learner to be able to
manipulate the environment.  Research methodologists distinguish between
experimental and non-experimental methods.  Non-experimental methods
rely on correlation of observations - so that patterns can be described
but causation cannot be ascribed.  Experimental methods involve assigning
manipulations to the environment in a randomised way - so that much stronger
statements can be made about the workings of the world (assuming your
assumptions hold :-).  Being able to manipulate the environment allows
the experimenter to drag information out of the environment at a higher
rate if the experimenter is in relative ignorance.  Cosmologists use
non-experimental methods - BUT they have to apply a lot of prior
knowledge that has been validated in other ways.

The other point I want to make is that typical connectionist categorisers
do not have a model of the input environment as an external reality.
Assume that the input signals come from a sensor that imposes some kind
of gross distortion.  The net learns to categorise the distorted signal
as it stands with no separate models of the environment and the distortion.
In order to develop a concept that  is closer to the human notion of some
external reality, the system has to be able to factor its model into an
environmental part and a perceptual part.  This can't be done by pure
observation, it needs some ability to interact with the environment,
even if only by panning the camera across the scene.