preprint: Functional Optimisation in Neural Networks

[Renato Vicente]

Title: Functional Optimisation of Online Algorithms in Neural Networks
       (submitted to J. Phys. A: Math. Gen.)
	
Authors: Renato Vicente and Nestor Caticha.

Available at : http://www.fma.if.usp.br/~rvicente/nnsp.html
	       http://xxx.lanl.gov  (cond-mat/9706015)

Abstract:

We study the online dynamics of learning in fully connected
soft committee machines in the student-teacher scenario. The locally optimal
modulation function, which determines the learning algorithm, is obtained
from a variational argument in such a manner as to maximise the average
generalisation error decay per example. Simulations results for the
resulting algorithm are presented for a few cases. The symmetric phase  
plateaux are found to be vastly reduced in comparison to those found when
online backpropagation algorithms are used. A discussion of the
implementation of these ideas as practical algorithms is given.

Comments are welcome.

Renato Vicente

Instituto de Fisica,
Universidade de Sao Paulo, Brazil