PhD thesis available

[Jonas Sjoberg]

My PhD thesis with the title

NON-LINEAR SYSTEM IDENTIFICATION WITH NEURAL NETWORKS

is available by FTP or WWW. It contains 223 pages and it is stored as
compressed postscript. (3.6 Mbyte uncompressed, 1.2 Mbyte compressed).


________________________________________________________________________________
Jonas Sjo"berg				
Dept. of El. Engineering		
University of Linkoping			Telefax: +46-13-282622, or +46-13-139282
S-581 83 Linko"ping			E-Mail: sjoberg at isy.liu.se 
Sweden
________________________________________________________________________________


Anonymous FTP: joakim.isy.liu.se or 130.236.24.1
directory: pub/Misc/NN/
file : PhDsjoberg.ps.Z

WWW: file://joakim.isy.liu.se/pub/Misc/NN/
file : PhDsjoberg.ps.

Abstract:

This thesis addresses the non-linear system identification problem,
and in particular, investigates the use of neural networks in system
identification.  An overview of different possible model structures is
given in a common framework. A nonlinear structure is described as the
concatenation of a map from the observed data to the regressor, and a
map from the regressor to the output space. This divides the model
structure selection problem into two problems with lower complexity:
that of choosing the regressor and that of choosing the non-linear map.

The possible choices for the regressors consists of past inputs and
outputs, and filtered versions of them. The dynamics of the model
depends on the choice of regressor, and families of different model
structures are suggested based on analogies to linear black-box
models. State-space models are also described within this common
framework by a special choice of regressor. It is shown that
state-space models which have no parameters in the state update
function can be viewed as an input-output model preceded by a
pre-filter.  A parameterized state update function, on the other hand,
can be seen as a data driven regressor selector.  The second step of
the non-linear identification is the mapping from the regressor to the
output space. It is often advantageous to try some intermediate
mappings between the linear and the general non-linear mapping. Such
non-linear black-box mappings are discussed and motivated by
considering different noise assumptions.

The validation of a linear model should contain a test for
non-linearities and it is shown that, in general, it is easy to detect
non-linearities. This implies that it is not worth spending too much
energy searching for optimal non-linear validation methods for a
specific problem. Instead the validation method should be chosen so
that it is easy to apply. Two such methods, based on polynomials and
neural nets, are suggested. Further, two validation methods, the
correlation-test and the parametric F-test, are investigated. It is
shown that under certain conditions these methods coincide.

Parameter estimates are usually based on criterion minimization. In
connection with neural nets it has been noted that it is not always
optimal to try to find the absolute minimum point of the criterion.
Instead a better estimate can be obtained if the numerical search for
the minimum is prematurely stopped. A formal connection between this
stopped search and regularization is given. It is shown that the
numerical minimization of the criterion can be view as a
regularization term which is gradually turned to zero. This closely
connects to, and explains, what is called overtraining in the neural
net literature.