Supervised learning

[Vasant Honavar]

A number of "supervised" learning tasks can be formulated as
learning tasks in which the "feedback" comes from a different
sensory modality - e.g., the feedback for a language learning may 
come from a visual scene. Learning in the two modalities may
be initiated through a process of "bootstrapping". 
Feedback can be viewed as coming from simply another input stream.
Instead of working with one input stream (as is typically the case
with most backprop-like schemes), the system learns by forming 
associations between two or more input streams. In this case,
it is hard to argue that the learning is any more "supervised"
than in any of the so-called "unsupervised" methods. This is 
conceptually quite similar to the self-supervised methods 
(Hinton, 1987; "Connectionist Learning Procedures").

It therefor appears useful to view the various learning schemes as
part of a continuum: Schemes requiring very specific feedback lie
(e.g., the desired output) on one extreme; those requiring no
feedback (all of learning is simply storing some abstractions of the
input) lie on the other extreme; and a host of schemes using various 
rather diffuse and non-specific forms of feedback (and possibly feedback
that is hidden in the form of local objective functions, information-
theoretic "constraints", the locus of interactions between units, etc.)
lie in the middle.

Vasant Honavar