Cross-entropy error

[homxb!solla@research.att.com]

A detailed discussion of cross-entropy error measure for back propagation, 
and a comparative study of its merits relative to the more commonly used 
quadratic measure is to be found in "Accelerated Learning in Layered Neural 
Networks" by S.A. Solla, E. Levin, and M. Fleisher. The paper has appeared 
in "Complex Systems", Vol. 2, 1988. 

Two other relevant references to the use of such error function in the 
context of supervised learning are:

E.B. Baum and F. Wilczek, "Supervised Learning of Probability Distributions 
by Neural Network" in "Neural Information Processing Systems", ed. by D. 
Anderson (AIP, New York, 1988)

J.J. Hopfield, "Leraning Algorithms and Probability Distributions in Feed-
forward and Feed-back Networks", Proc. Natl. Acad. Sci. USA, Vol. 84 ,1988, 
p. 8429-8433.

Sara A. Solla 
AT&T Bell Laboratories 
solla at homxb.att.com