new paper on Multi-model Algorithm for Parameter Estimation of Time-varying Nonlinear Systems

[Thanasis Kehagias]

The following paper will appear in AUTOMATICA. While it is not strictly
about neural networks, the presented analysis of credit assignment
convergence, for a multimodel scheme, may be of interest for people
working with modular neural networks, mixtures of experts and so on.

Title: A Multi-model Algorithm for Parameter Estimation of Time-varying
Nonlinear Systems
Authors: V. Petridis and Ath. Kehagias
Source: Automatica (to appear)
Link: http://skiron.control.ee.auth.gr/~kehagias/97epeke.htm

Abstract: Many methods have been developed to solve the problem of
parameter stimation for dynamical systems (Ljung, 1987). Of particular
interest is the case of on-line algorithms which are used to estimate
time-varying parameters. Here we  present such an algorithm which
assumes a nonlinear dynamical system. The system is time-varying: its
parameter changes values according to a Markovian model switching
mechanism. The algorithm starts with a finite number of models, each
corresponding to one of the parameter values, and selects the
``phenomenologically best'' parameter value; namely the one which
produces the best fit to the observed behavior of the system. Our
algorithm is related to the Partition Algorith (PA) presented in
(Hilborn & Lainiotis, 1969; Lainiotis, 1971; Lainiotis & Plataniotis,
1994; Sims, Lainiotis & Magill, 1969). PA is suitable for the parameter
estimation of a linear dynamical system with Gaussian noise in the input
and output; no provision is made for model switching. Under these
assumptions, an algorithm is developed for exact computation of the
models' posterior probabilities; these are used for Maximum a Posteriori
(MAP) estimation of the unknown parameter. This method has been used
extensively in a number of applications, including parameter estimation
and system identification (Kehagias, 1991; Lainiotis & Plataniotis,
1994; Petridis, 1981). Our algorithm is more general than the PA: it
applies to nonlinear systems and requires no  probabilistic assumptions
regarding the noise. Furthermore, while there are several convergence
studies of the PA without a switching mechanism (Anderson & Moore, 1979;
Kehagias, 1991; Tugnait, 1980), as far as we know, the analysis
presented here is the first one that handles the Markovian switching
assumption. A rigorous convergence analysis is also presented.


Thanasis Kehagias, 
Research Associate, Dept. of Electrical and Computer Eng, Aristotle Un.,
Thessaloniki
Ass. Prof., Dept. of Mathematics and Computer Sci., American College of
Thessaloniki

http://skiron.control.ee.auth.gr/~kehagias/index.htm