Recent abstracts

[Eduardo Sontag]

I enclose abstracts of some recent technical reports.  (Ignore the skip
numbers; the rest are not in any manner related to NN's.)
***Any suggestions as to which journal to send 88-08 to*** would be highly
appreciated.  (There don't appear to be any journals geared towards very
mathematical papers in NN's, it would seem.)
________________________________________________________________________
       ABSTRACTS OF SYCON REPORTS  
    Rutgers Center for Systems and Control 
    Hill Center, Rutgers University, New Brunswick, NJ 08903 
    E-mail:    sycon at fermat.rutgers.edu  
 
 [88-01]
   Two algorithms for the Boltzmann machine: description, implementation,
and a preliminary comparative study of performance , Lorien Y. Pratt and
H  J. Sussmann, July 88.
 
This report compares two algorithms for learning in neural networks: the
Boltzmann and modified Boltzmann machines.  The Boltzmann machine has been
extensively studied in the past; we have recently developed the modified
Boltzmann machine. We present both algorithms and discuss several
considerations which must be made for their implementation. We then give a
complexity analysis and preliminary empirical comparison of the two
algorithms' learning ability on a benchmark problem.  For this problem, the
modified Boltzmann machine is shown to learn slightly slower than the
Boltzmann machine.  However, the modified algorithm does not require the user
to build an annealing schedule to be used for training.  Since building this
schedule constitutes a significant amount of the engineering time for the
Boltzmann algorithm, we feel that our modified algorithm may be superior to
the classical one. Since we have not yet performed a rigorous comparison of
the two algorithms' performance, it may also be possible to optimize the
parameters to the modified algorithm so that the learning speed is comparable
to the classical version.

 [88-02]
   Some remarks on the backpropagation algorithm for neural net learning ,
Eduardo D. Sontag, July 88.  (13 pages.)
 
This report contains some remarks about the backpropagation method for neural
net learning.  We concentrate in particular in the study of local minima of
error functions and the growth of weights during learning.

 [88-03]
   On the convergence of learning algorithms for Boltzmann machines ,
H  J. Sussmann, July 88.  (46 pages.)
 
We analize a learning algorithm for Boltzmann machines, based on the usual
alternation between ``learning'' and ``hallucinating'' phases.  We prove
rigorously that, for suitable choices of the parameters, the evolution of the
weights follows very closely, with very high probability, an integral
trajectory of the gradient of the likelihood function whose global maxima are
exactly the desired weight patterns.  An abridged version of this report will
appear in the Proceedings of the 27th IEEE Conference on Decision and Control,
December 1988.

 [88-08]
   Backpropagation can give rise to spurious local minima even for networks
without hidden layers , Eduardo D. Sontag and H  J. Sussmann, Sept 88.
(15 pages.)
 
We give an example of a neural net without hidden layers and with a sigmoid
transfer function, and a corresponding training set of binary vectors, for
which the sum of the squared errors, regarded as a function of the weights,
has a local minimum which is not a global minimum