Abstract

[M. Niranjan]

Here is an extended summary of a Tech report now available. Apologies for
the incomplete de-TeXing.

niranjan

PS: Remember, reprint requests should be sent to
    "niranjan at dsl.eng.cam.ac.uk" and NOT "connectionists at q.cs.cmu.edu"

=============================================================================


	     NEURAL NETWORKS AND RADIAL BASIS FUNCTIONS
		IN CLASSIFYING STATIC SPEECH PATTERNS

		   Mahesan Niranjan & Frank Fallside

		      CUED/F-INFENG/TR 22

University Engineering Department
Cambridge, CB2 1PZ, England
Email: niranjan at dsl.eng.cam.ac.uk

SUMMARY

This report compares the performances of three non-linear pattern classifiers
in the recognition of static speech patterns. Two of these classifiers are
neural networks (Multi-layered perceptron and the  modified Kanerva model
(Prager & Fallside, 1988)). The third is the method of radial basis functions
(Broomhead & Lowe, 1988).

The high performance of neural-network based pattern classifiers shows
that simple linear classifiers are inadequate to deal with complex patterns
such as speech. The Multi-layered perceptron (MLP) gives a mechanism to
approximate an arbitrary classification boundary (in the feature space) to a
desired precision. Due to this power and the existence of a simple learning
algorithm (error back-propagation), this technique is in very wide use
nowadays.

The modified Kanerva model (MKM) for pattern classification is derived from
a model of human memory (Kanerva, 1984). It attempts to take advantage of
certain mathematical properties of binary spaces of large dimensionality.
The modified Kanerva model works with real valued inputs. It compares an
input feature vector with a large number of randomly populated `location
cells' in the input feature space; associated with every cell is a `radius'.
Upon comparison, the cell outputs value 1 if the input vector lies within
a volume defined by the radius; its output is zero otherwise. The discrimi-
nant function of the Modified Kanerva classifier is a weighted sum of these
location-cell outputs. It is trained by a gradient descent algorithm.

The method of radial basis functions (RBF) is a technique for non-linear
discrimination. RBFs have been used by Powell (1985) in  multi-variable
interpolation. The non-linear discriminant function in this method is of the
form,

g( x) = sum_j=1^m lambda_j phi (||x - x_j||)

Here, x is the feature vector. lambda_j  are weights associated with each of
the given training examples x_j. phi is a  kernel function defining the
range of influence of each data point on the class boundary. For a particular
choice of the phi function, and a set of training data {x_j,f_j}, j=1,...,N,
the solution for the lambda_j s is closed-form. Thus this technique is
computationally simpler than most neural networks. When used as a non-
parametric technique, each computation at classification stage involves the
use of all the training examples. This, however, is not a disadvantage since
much of this computing can be done in parallel.

In this report, we compare the performance of these classifiers on speech
signals.  Several techniques similar to the method of radial basis functions
are reviewed. The properties of the class boundaries generated by the MLP,
MKM and RBF are derived on simple two dimensional examples and an experimental
comparison with speech data is given.

============================================================================