PhD Thesis

[avner priel]

Dear Connectionists,

My PhD thesis is now available from the following URL:

	http://www.math.tau.ac.il/~priel/papers.html

Below please find the abstract.

Best wishes,

	Priel Avner.


-----------------------------------------------------
  Priel Avner  < priel at math.tau.ac.il >           
               < http://www.math.tau.ac.il/~priel >
  School of Mathematical Sciences, 
  Tel-Aviv University,                         
  Israel.

-----------------------------------------------------------------------
-----------------------------------------------------------------------


      "Dynamic and Static Properties of Neural Networks with FeedBack"

				Ph.D. Thesis

				 Avner Priel
			    Department of Physics  
		         Bar-Ilan University, Israel.



ABSTRACT:

This thesis describes analytical and numerical study of time series 
generated by a special type of recurrent neural networks,
a continuous-valued feed-forward network in which the next input 
vector is determined from past output values. 
The topics covered in this work include the analysis of the sequences
generated by the network in the stable and unstable regimes of the
parameter space, the effect of an additive noise on the long-term properties 
of the network and the ability of the model to capture the rule of a
long-range correlated sequence.

The asymptotic solutions of the sequences generated by the model in the
stable regime are found analytically for various architectures, transfer
functions and choice of the weights. We find that the generic solution is a 
quasi-periodic attractor (excluding the cases where the solution is a 
fixed point). We find a hierarchy among the complexity of time series 
generated by different architectures; more hidden units can generate
higher dimensional attractors. The relaxation time from an arbitrary
initial condition to the vicinity of the asymptotic attractor is studied
for the case of a perceptron and a two-layered perceptron. In both cases,
the relaxation time scales linearly with the size of the network.

Although networks with monotonic, as well as non-monotonic, transfer
functions are capable of generating chaotic sequences, the unstable
regions of the parameter space exhibit different features. Non-monotonic
functions can produce robust chaos, whereas monotonic functions
generate fragile chaos only. In the case of non-monotonic functions, 
the number of positive Lyapunov exponents increases as a function of one 
of the free parameters in the model; hence, high dimensional chaotic 
attractors can be generated. We study also a combination 
of monotonic and non-monotonic functions. 

The stability of the asymptotic results obtained for the model is tested
by analysing the effect of an additive noise introduced in the output of
the network. A single attractor in the presence of noise is broadened.
The phase of a noisy model diffuses w.r.t.\ a noise-free model with a
diffusion constant which is inversely proportional to the size of the
network; hence, phase coherence is maintained for a time length that is
proportional to the network's size. 
When the network has more than a single possible attractor, they become
meta-stable in the presence of noise. We study the properties of an
important quantity - the mean first passage time, and derive a relation
between the size of the network, the distance from the bifurcation point
and the mean first passage time to escape from the basin of attraction.

The last subject we address concerns the capability of our model to learn
the rule of a sequence obeying a power-law correlation function. An ensemble
of long sequences is generated and used to train the network. The trained
networks are then used to generate a subsequent sequences. It is found
that the generated sequences have a similar power-law correlation function as 
the training sequences. By studying the properties of the trained networks, we 
conclude that the correlation function of the weight matrix should be
dominated by vertical power-law correlations in order to generate
long-range correlated sequences. This conclusion is verified by numerical
simulations. Analysis of the mean-field approximation of the correlation
function leads to the same qualitative conclusion regarding the weight
matrix.