Software + Papers

[J.F. Gomes De Freitas]

Dear colleagues

You can find the following papers:

1 Sequential Monte Carlo Methods for Optimisation of Neural Network 
Models (a similar version to appear in Neural Computation). 

2 Nonlinear State Space Estimation with Neural Networks and
the EM algorithm (possibly to appear in a special issue of VLSI Signal 
Processing Systems).

and the Matlab software at my Cambridge web site: 
http://svr-www.eng.cam.ac.uk/~jfgf/software.html

I'd be grateful for feedback. The abstracts follow:

1 Sequential Monte Carlo Methods for Optimisation of Neural Network Models:
We discuss a novel strategy for training neural networks using sequential 
Monte Carlo algorithms and propose a new hybrid gradient descent/sampling 
importance resampling algorithm (HySIR). In terms of both computational 
time and accuracy, the hybrid SIR is a clear improvement over 
conventional sequential Monte Carlo techniques.
The new algorithm may be viewed as a global optimisation strategy,
which allows us to learn the probability distributions of the network 
weights and outputs in a sequential framework. It is well suited to 
applications involving on-line, nonlinear and non-Gaussian signal 
processing. We show how the new algorithm outperforms extended Kalman 
filter training on several problems. In particular,
we address the problem of pricing option contracts, traded
in financial markets. In this context, we are able to estimate the
one-step-ahead probability density functions of the options prices.

2 Nonlinear 
State Space Estimation with Neural Networks and the EM algorithm: 
In this paper, we derive an EM algorithm for nonlinear state space 
models. We use it to estimate jointly the neural network weights, the 
model uncertainty and the noise in the data. In the E-step we apply a 
forward-backward Rauch-Tung-Striebel smoother to compute the network 
weights. For the M-step, we derive expressions to compute the model 
uncertainty and the measurement noise. We find that the method is 
intrinsically very powerful, simple, elegant and stable. 

Best wishes
Nando


_______________________________________________________________________________

JFG de Freitas (Nando)
Speech, Vision and Robotics Group
Information Engineering
Cambridge University
CB2 1PZ England
http://svr-www.eng.cam.ac.uk/~jfgf        
Tel (01223) 302323 (H)                 
    (01223) 332754 (W)
_______________________________________________________________________________