Modelling of Nonlinear Systems

[J.R.Chen@durham.ac.uk]

    IS INPUT_OUTPUT EQUATION A UNIVERSAL MODEL OF NONLINEAR SYSTEM ?

About two weeks ago, Kenji Doya announced a paper "Universality of 
Fully-Connected Recurrent Neural Networks" on this mail list, which 
showed that if all the state variables are available then any discrete or 
continuous-time dynamical system can be modeled by a fully-connected 
discrete or continuous-time recurrent network respectively, provide the network 
consists of enough units. This is interesting. However in the real situation, 
it is more likely that the number of observable  variables is less than the 
degree of freedom of the dynamical system. It could be that only 
one output signal is available for measurement. So the question is if only 
input signal and one output signal is available from a dynamical system, is it 
possible to reconstructe the original dynamics of the system? This problem 
has been well studied for linear systems, and the theory is well established. 
For the nonlinear systems, it seems is still a partially open question.    

There has been a lot of publications on using recurrent neural networks, 
MLP nets or whatever other nets to model nonlinear time-series or for 
nonlinear system identification. This kind of approach is based on an   
assumption that a nonlinear system can be modelled by an input-output 
recursive equation just like a linear system can be modelled by a ARMA 
model. A typical argument could be like this 

"Because the n variables {X_k(t)} satisfy a set of first-order differential 
equation, successive differentiation in time reduces the problem to a single 
(general highly nonlinear) differential equation of nth order for one of 
these variables"

One can say something similar for discrete systems. Sounds it is quite 
straight forward. Actually, in most equation specific cases, it do works that 
way. However obviously this is not a rigorous proof. To my knowledge, the 
most rigorous results on this problem is presented in F. Takens[1], 
I.J.Leontaritis and S.A.Billings[2].  [1] mainly discusses autonomous 
systems and is wildly referenced. It is the theoriatical fundation of almost 
all the work on chaotic time-series modelling or prediction. In [2] it has 
been proved under some conditions that a discrete nonlinear system can be 
represented by a input-output recursive equation in a restricted region of 
operation around the zero equilibrium point. I don't know is there any 
global results exist. If not, the queation would be is this mainly a difficult 
of mathematics,  or it would be more fundamental? One might to speculate 
that for a generic nonlinear dynamical system, there might be no unique 
input-output recursive equation representation, it may need a set of 
equations for different operation regions in the state space.  If this is true, 
the modelling of nonlinear dynamical systems with input-output equation 
has to be based on on-line approach. The parameters have to be updated 
quick enough to follow the moving of operation point. The off-line 
modelling or identification may have convergence problem.

[1] F. Takens "Detecting strange attractors in turbulence" in Springer 
Lecture Notes in Mathematics  Vol.893  p366  edited by D.A.Rand and 
L.S.Young   1981

[2] I.J.Leontaritis and S.A.Billings "Input-output parametric models for 
non-linear systems" Part-1 and Part-2  INT.J.CONTROL.  Vol.41 No.2  
pp303-328 and pp329-344. 1985.


     J R Chen

    SECS
    University of Durham, UK