Three TRs and software on on-line learning and applications

[Nik Kasabov]

Dear colleagues,

Three Technical Reports and software functions in MATLAB on evolving
connectionist systems for on-line learning and their applications for
on-line adaptive speech recognition and dynamic time series prediction are
available from:

http://divcom.otago.ac.nz/infoscience/kel/CBIIS.html (software/EFuNN)

regards,
Nik Kasabov
--------------------------------------------
Prof. Dr. Nikola (Nik) Kasabov
Department of Information Science
University of Otago,P.O. Box 56,Dunedin
New Zealand, email: nkasabov at otago.ac.nz
phone:+64 3 479 8319, fax:  +64 3 479 8311
http://divcom.otago.ac.nz:800/infosci/Staff/NikolaK.htm
--------------------------------------------

TR99/02
N.Kasabov, Evolving Connectionist Systems for On-line, Knowledge-based
Learning: Principles and Applications, TR99/02, Department of Information
Science, University of Otago, New Zealand  

Abstract. The paper introduces evolving connectionist systems (ECOS) as an
effective approach to building on-line, adaptive intelligent systems. ECOS
evolve through incremental, hybrid (supervised/unsupervised), on-line
learning. They can accommodate new input data, including new features, new
classes, etc. through local element tuning. New connections and new neurons
are created during the operation of the system. The ECOS framework is
presented and illustrated on a particular type of evolving neural networks -
evolving fuzzy neural networks (EFuNNs). EFuNNs can learn spatial-temporal
sequences in an adaptive way through one pass learning. Rules can be
inserted and extracted at any time of the system operation. The
characteristics of ECOS and EFuNNs are illustrated on several case studies
that include: adaptive pattern classification; adaptive, phoneme-based
spoken language recognition; adaptive dynamic time-series prediction;
intelligent agents. 

----------------------------------------------------------------------------
----
TR99/03
N.Kasabov, M.Watts, Spatial-Temporal Adaptation in Evolving Fuzzy Neural
Networks for On-line Adaptive Phoneme Recognition, TR99/03, Department of
Information Science, University of Otago, New Zealand

Abstract. The paper is a study on the spatial-temporal characteristics  of
evolving fuzzy neural network systems (EFuNNs)for on-line adaptive learning.
These characteristics  are important for the task of adaptive, speaker
independent spoken language recognition, where new pronunciations and new
accents need to be learned in an on-line, adaptive mode. Experiments with
EFuNNs, and also with multi-layer perceptrons, and fuzzy neural networks
(FuNNs), conducted on the whole set of 43 New Zealand English phonemes, show
the superiority and the potential of EFuNNs when used for the task. Spatial
allocation of nodes and their aggregation in EFuNNs allow for similarity
preserving and similarity observation within one phoneme data and across
phonemes, while subtle temporal variations within one phoneme data can be
learned and adjusted through temporal feedback connections. The experimental
results support the claim that spatial-temporal organisation in EFuNNs can
lead to a significant improvement in the recognition rate especially for the
diphthong and the vowel phonemes in English, which in many cases are
problematic for a system to learn and adjust in an on-line, adaptive way.


----------------------------------------------------------------------------
----
TR99/04
N.Kasabov and Q.Song, Dynamic Evolving Fuzzy Neural Networks with
'm-out-of-n' Activation Nodes for On-line Adaptive Systems, TR99/04,
Department of Information Science, University of Otago, New Zealand   

Abstract. The paper introduces a new type of evolving fuzzy neural networks
(EFuNNs), denoted as  mEFuNNs, for on-line learning and their applications
for dynamic time series analysis and prediction. At each time moment the
output vector of a mEFuNN is calculated based on the m-most activated rule
nodes. Two approaches are proposed: (1) using weighted fuzzy rules of
Zadeh-Mamdani type; (2) using Takagi-Sugeno fuzzy rules that utilise
dynamically changing  and adapting values for the inference parameters. It
is proved that the mEFuNNs can effectively learn complex temporal sequences
in an adaptive way and outperform  EFuNNs, ANFIS and other connectionist and
hybrid connectionist models. The characteristics of the mEFuNNs are
illustrated on two bench-mark dynamic time series data, as well as on two
real case studies for on-line adaptive control and decision making.
Aggregation of rule nodes in evolved mEFuNNs can be achieved through fuzzy
C-means clustering algorithm which is also illustrated on the bench mark
data sets. The regularly trained and aggregated in an on-line,
self-organised mode mEFuNNs perform as well, or better, than the mEFuNNs
that use fuzzy C-means clustering algorithm for off-line rule node
generation on the same data set. 

-------------------------------------------------------------------