non-MSE objective function for backprop

[John.Hampshire@SPEECH2.CS.CMU.EDU]

*********  FOR CONNECTIONISTS ONLY - PLEASE DO NOT FORWARD  ***********
****************  TO OTHER BBOARDS/ELECTRONIC MEDIA *******************


A NOVEL OBJECTIVE FUNCTION FOR IMPROVED CLASSIFICATION PERFORMANCE
     IN TIME-DELAY NEURAL NETS USED FOR PHONEME RECOGNITION

            J. B. Hampshire II       A. H. Waibel
  	         Carnegie Mellon University

We have been working on an alternative objective function
to the mean-squared-error (MSE) objective function typically
used in backpropagation.  Our alternative, which we term the
classification figure-of-merit (CFM), forms a mathematical assessment
of the *relative* activations of all output nodes of a backprop
network used as a classifier.  The objective function has a number
of unique characteristics; chief among these are 

1.  its formation of internal representations that consistently
    differ substantially from those of the MSE objective function

2.  its immunity to "over-learning" (i.e., the process by which
    MSE classifiers can be trained so much that they begin to
    key on "idiosyncratic" features of the training set that are
    not representative of the ensemble from which the training
    set was drawn.  As a result, over training actually results in
    degraded classification performance on a disjoint test set.)


While classification performance of the CFM objective function is
equivalent to that of the MSE objective function, results
from the two classifiers can be combined to reduce by a median 24%
the number of misclassifications made by the MSE classifier alone.
This equates to single and multi-speaker /b, d, g/ recognition rates
that consistently exceed 98%.

A preliminary paper is available on our results of applying
the CFM to phoneme recognition using Time-Delay Neural Nets now,
but if you want to wait another two weeks, you can get the NEW! IMPROVED!
full-fledged technical report.

If you absolutely can't wait to get your hands on this stuff, send
your mailing address and something to the effect of, "send me the
CFM paper."

If, on the other hand, you want to see a more thorough analysis,
send your mailing address and say, "send me the CFM tech report 
(CMU-CS-89-118) in two weeks."

In either case, send your request directly to 

hamps at speech2.cs.cmu.edu

***** DO NOT USE THE REPLY COMMAND IN YOUR MAILER *****


*********  FOR CONNECTIONISTS ONLY - PLEASE DO NOT FORWARD  ***********
****************  TO OTHER BBOARDS/ELECTRONIC MEDIA *******************