New Book on modular NN and time series
Thanasis Kehagias
kehagias at egnatia.ee.auth.gr
Tue Sep 29 13:45:08 EDT 1998
NEW BOOK on Modular Neural Networks and Time Series
The following book has just been published by Kluwer Academic Publishers.
TITLE: Predictive Modular Neural Networks: Applications to Time Series
AUTHORS: V. Petridis and Ath. Kehagias
PUBLISHER: Kluwer Academic Publishers, Boston
YEAR: 1998
ISBN: 0-7923-8290-0
This book will be of interest to connectionists, machine learning
researchers, statisticians, control theorists and perhaps also to
researchers in biological and medical informatics, researchers in
econometrics and forecasting as well as psychologists. It can be ordered
from Kluwer's web site at
http://www.wkap.nl/book.htm/0-7923-8290-0
The general subject of the book is the application of modular neural
networks (in another terminology: multiple models) to problems of time
series classification and prediction. The problem of system identification
is also treated as a time series problem. We consider both supervised
learning of labelled TS data and unsupervised learning of unlabelled TS
data. We present a general framework for the design of PREDICTIVE MODULAR
algorithms and provide a rigorous convergence analysis for both the
supervised and unsupervised cases. We also present the application of the
above algorithms to three real world problems (encephalogram
classification, electric load prediction and waste water plant parameter
etsimation). Finally, we provide an extensive bibliography of modular and
multiple models methods and discuss the connections between such methods
which have appeared in the neural networks as well as in other research
communities.
More info about the book can be found at
http://www.wkap.nl/book.htm/0-7923-8290-0
or at
http://skiron.control.ee.auth.gr/~kehagias/thn/thn02b01.htm
-------------------------------------------------------------
TABLE OF CONTENTS
1. Introduction
1.1 Classification, Prediction and Identification: an Informal Description
1.2 Part I: Known Sources
1.3 Part II: Applications
1.4 Part III: Unknown Sources
1.5 Part IV: Connections
PART I Known Sources
2. PREMONN Classification and Prediction
2.1 Bayesian Time Series Classification
2.2 The Basic PREMONN Classification Algorithm
2.3 Source Switching and Thresholding
2.4 Implementation and Variants of the PREMONN Algorithm
2.5 Prediction
2.6 Experiments
2.7 Conclusions
3. Generalizations of the Basic PREMONN
3.1 Predictor Modifications
3.2 Prediction Error Modifications
3.3 Credit Assignment Modifications
3.4 Markovian Source Switching
3.5 Markovian Modifications of Credit Assignment Schemes
3.6 Experiments
3.7 Conclusions
4. Mathematical Analysis
4.1 Introduction
4.2 Convergence Theorems for Fixed Source Algorithms
4.3 Convergence Theorem for a Markovian Switching Sources Algorithm
4.4 Conclusions
5. System Identification by the Predictive Modular Approach
5.1 System Identification
5.2 Identification and Classification
5.3 Parameter Estimation: Small Parameter Set
5.4 Parameter Estimation:\ Large Parameter Set
5.5 Experiments
5.6 Conclusions
PART II Applications
6. Implementation Issues
6.1 PREMONN Structure
6.2 Prediction
6.3 Credit Assignment
6.4 Simplicity of Implementation
7. Classification of Visually Evoked Responses
7.1 Introduction
7.2 VER Processing and Classification
7.3 Application of PREMONN Classification
7.4 Results
7.5 Conclusions
8. Prediction of Short Term Electric Loads
8.1 Introduction
8.2 Short Term Load Forecasting Methods
8.3 PREMONN Prediction
8.4 Results
8.5 Conclusions
9. Parameter Estimation for and Activated Sludge Process
9.1 Introduction
9.2 The Activated Sludge Model
9.3 Predictive Modular Parameter Estimation
9.4 Results
9.5 Conclusions
PART III Unknown Sources
10. Source Identification Algorithms
10.1 Introduction
10.2 Source Identification and Data Allocation
10.3 Two Source Identification Algorithms
10.4 Experiments
10.5 A Remark about Local Models
10.6 Conclusions
11. Convergence of Parallel Data Allocation
11.1 The Case of Two Sources
11.2 The Case of Many Sources
11.3 Conclusions
12. Convergence of Serial Data Allocation
12.1 The Case of Two Sources
12.2 The Case of Many Sources
12.3 Conclusions
PART IV Connections
13. Bibliographic Remarks
13.1 Introduction
13.2 Neural Networks
Combination of Specialized Networks252
Ensembles of Networks
Mixtures of Experts
RBF and Related Networks
Trees
13.3 Statistical Pattern Recognition
13.4 Econometrics and Forecasting
13.5 Fuzzy Systems
13.6 Control Theory
13.7 Statistics
14. Epilogue
Appendix: Mathematical Concepts
References
Index
-------------------------------------------------------------
The book's PREFACE
The subject of this book is predictive modular neural networks and their
application to time series problems: classification, prediction and
identification. The intended audience is researchers and
graduate students in the fields of neural networks, computer science,
statistical pattern recognition, statistics, control theory and
econometrics. Biologists, neurophysiologists and medical engineers
may also find this book interesting.
In the last decade the neural networks community has shown intense interest
in both modular methods and time series problems. Similar interest has been
expressed for many years in other fields as well, most notably in
statistics, control theory, econometrics etc. There is a considerable
overlap (not always recognized) of ideas and methods between these fields.
Modular neural networks come by many other names, for instance multiple
models, local models and mixtures of experts. The basic idea is to
independently develop several ``subnetworks'' (modules), which may perform
the same or related tasks, and then use an ``appropriate'' method
for combining the outputs of the subnetworks. Some of the expected
advantages of this approach (when compared with the use of ``lumped'' or
``monolithic'' networks) are: superior performance, reduced development
time and greater flexibility. For instance, if a module is removed from the
network and replaced by a new module (which may perform the same task more
efficiently), it should not be necessary to retrain the aggregate network.
In fact, the term ``modular neural networks'' can be rather vague. In its
most general sense, it denotes networks which consist of simpler
subnetworks (modules). If this point of view is taken to
the extreme, then every neural network can be considered to be modular, in
the sense that it consists of neurons which can be seen as elementary
networks. We believe, however, that it is more profitable to think of a
continuum of modularity, placing complex nets of very simple neurons
at one end of the spectrum, and simple nets of very complex neurons at the
other end.
We have been working along these lines for several years and have developed
a family of algorithms for time series problems, which we call PREMONN's
(i.e. PREdictive MOdular Neural Networks). Similar algorithms and systems
have also been presented by other authors, under
various names. We will generally use the acronym PREMONN to refer to our
own work and retain ``predictive modular neural networks'' as a generic term.
This book is divided in four parts. In Part I we present some of our work
which has appeared in various journals such as IEEE Transactions on Neural
Networks, IEEE Transactions on Fuzzy Systems, Neural Computation, Neural
Networks etc. We introduce the family of PREMONN algorithms. These
algorithms are appropriate for online time series classification,
prediction and identification. We discuss these algorithms at an informal
level and we also analyze mathematically their convergence properties.
In Part II we present applications (developed by ourselves and other
researchers) of PREMONNs to real world problems. In both these parts a
basic assumption is that models are available to describe the input /
output behavior of the sources generating the time series of
interest. This is the known sources assumption.
In Part III we remove this assumption and deal with time series generated
by completely unknown sources. We present algorithms which operate online
and discover the number of sources involved in the generation of a time
series and develop input/ output models for each source. These source
identification algorithms can be used in conjunction with the
classification and prediction algorithms of Part I. The results of Part III
have not been previously published.
Finally, in Part IV we briefly review work on modular and multiple models
methods which has appeared in the literature of neural networks,
statistical pattern recognition, econometrics, fuzzy systems, control
theory and statistics. We argue that there is a certain unity of themes and
methods in all these fields and provide a unified interpretation of the
multiple models idea. We hope that this part will prove useful by pointing
out and elucidating similarities between the multiple models
methodologies which have appeared in several disparate fields.
Indeed, we believe that there is an essential unity in the modular
approach, which cuts across disciplinary boundaries. A good example is the
work reported in this book. While we present our work in ``neural''
language, its essential characteristic is the combination of simple
processing elements which can be combined to form more complex (and
efficient) computational structures. There is nothing exclusively neural
about this theme; it has appeared in all the above mentioned
disciplines and this is why we believe that a detailed literature search
can yield rich dividends in terms of outlook and technique cross
fertilization.
The main prerequisite for reading this book is the basics of neural network
theory (and a little fuzzy set theory). In Part I, the mathematically
involved sections are relegated to appendices, which may be left for a
second reading, or omitted altogether. The same is true of Part III:
convergence proofs (which are rather involved) are presented in appendices,
while the main argument can be followed quite independently of the
mathematics. Parts II and IV are nonmathematical. We have
also provided an appendix, which contains the basic mathematical concepts
used throughout the book.
___________________________________________________________________
Ath. Kehagias
--Assistant Prof. of Mathematics, American College of Thessaloniki
--Research Ass., Dept. of Electrical and Computer Eng. Aristotle Univ.,
Thessaloniki, GR54006, GREECE
--email: kehagias at egnatia.ee.auth.gr, kehagias at ac.anatolia.edu.gr
--web: http://skiron.control.ee.auth.gr/~kehagias/index.htm
More information about the Connectionists
mailing list