paper in neuroprose

[becker@ai.toronto.edu]

A postscript version of my PhD thesis has been placed in the neuroprose
archive.  It prints on 150 pages. The abstract is given below, followed by
retrieval instructions.


Sue Becker
email: becker at ai.toronto.edu

-----------------------------------------------------------------------------


 An Information-theoretic Unsupervised Learning Algorithm for Neural Networks



				   ABSTRACT

In the unsupervised learning paradigm, a network of neuron-like units is
presented an ensemble of input patterns from a structured environment, such as
the visual world, and learns to represent the regularities in that input. The
major goal in developing unsupervised learning algorithms is to find objective
functions that characterize the quality of the network's representation
without explicitly specifying the desired outputs of any of the units.
Previous approaches in unsupervised learning, such as clustering, principal
components analysis, and information-transmission-based methods, make minimal
assumptions about the kind of structure in the environment, and they are good
for preprocessing raw signal input. These methods try to model {\em all} of
the structure in the environment in a single processing stage.  The approach
taken in this thesis is novel, in that our unsupervised learning algorithms do
not try to preserve all of the information in the signal.  Rather, we start by
making strongly constraining assumptions about the kind of structure of
interest in the environment. We then proceed to design learning algorithms
which will discover precisely that structure.  By constraining what kind of
structure will be extracted by the network, we can force the network to
discover higher level, more abstract features. Additionally, the constraining
assumptions we make can provide a way of decomposing difficult learning
problems into multiple simpler feature extraction stages. We propose a class
of information-theoretic learning algorithms which cause a network to become
tuned to spatially coherent features of visual images. Under Gaussian
assumptions about the spatially coherent features in the environment, we have
shown that this method works well for learning depth from random dot
stereograms of curved surfaces.  Using mixture models of coherence, these
algorithms can be extended to deal with discontinuities, and to form multiple
models of the regularities in the environment.  Our simulations demonstrate
the general utility of the Imax algorithms in discovering interesting,
non-trivial structure (disparity and depth discontinuities) in artificial
stereo images. This is the first attempt we know of to model perceptual
learning beyond the earliest stages of low-level feature extraction, and to
model multiple stages of unsupervised learning.


-----------------------------------------------------------------------------


To retrieve from neuroprose:

unix> ftp cheops.cis.ohio-state.edu
Name (cheops.cis.ohio-state.edu:becker): anonymous
Password: (use your email address)
ftp> cd pub/neuroprose
ftp> get becker.thesis1.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for becker.thesis1.ps.Z (292385 bytes).
226 Transfer complete.
292385 bytes received in 13 seconds (22 Kbytes/s)
ftp> get becker.thesis2.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for becker.thesis2.ps.Z (366573 bytes).
226 Transfer complete.
366573 bytes received in 15 seconds (23 Kbytes/s)
ftp> get becker.thesis3.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for becker.thesis3.ps.Z (178239 bytes).
226 Transfer complete.
178239 bytes received in 9.2 seconds (19 Kbytes/s)
ftp> quit
221 Goodbye.
unix> uncompress becker*
unix> lpr becker.thesis1.ps
unix> lpr becker.thesis2.ps
unix> lpr becker.thesis3.ps