Mathematical analysis of unsupervised learning systems.

[Simone G.O. Fiori (An)]

Dear Colleagues,
I would like to announce the availability of three new papers devoted to the
mathematical analysis of unsupervised neural learning systems.

Sincerely,
Simone Fiori

Unsupervised Neural Learning on Lie Group
=============================================================================
Author: S. Fiori, Faculty of Engineering, University of Perugia (Italy)
Journal: International Journal of Neural Systems

Abstract:
The present paper aims at introducing the concepts and mathematical
details of unsupervised neural learning with orthonormality
constrains. The neural structures considered are single non-linear
layers and the learnable parameters are organized in matrices, as
usual, which gives the parameters spaces the geometrical structure of
the Euclidean manifold. The constraint of orthonormality for the
connection-matrices further restrict the parameters spaces to
differential manifolds such as the orthogonal group, the compact
Stiefel manifold and its extensions. For these reasons, the
instruments for characterizing and studying the behavior of learning
equations for these particular networks are provided by the
differential geometry of Lie groups. Although the considered class of
learning theories is very general, in the present paper special
attention is paid to unsupervised learning paradigms.

Download from: http://www.unipg.it/~sfr/publications/IJNS02.ps [46
pages, 565 KB]

Notes on Bell-Sejnowski PDF-Matching Neuron
=============================================================================
Author: S. Fiori, Faculty of Engineering, University of Perugia (Italy)
Journal: Neural Computation

Extended abstract:
Independent component analysis (ICA) is an emerging neural signal
processing technique that allows representing sets of signals as
linear combinations of statistically independent bases. In particular,
the results of recent investigations about the statistical properties
of natural images in relation to the properties of simple cells in V1,
suggest that these cells learn to form spatial filters that perform an
independent component analysis of the images.  One of the most
interesting aspects of the ICA theory proposed by Bell and Sejnowski
(1996) is the emerging ability of the neurons in the structural model
to align to the statistical distributions of the stimuli. Such
observation was successfully exploited in order to design different
learning rules for blind separation, blind deconvolution and
probability density function estimation.  

In the present paper we consider the basic Bell-Sejnowski class of
neuron models and recall the maximum-entropy adapting formulas. By
properly selecting a model in the class, that gives rise to tractable
mathematics, we are able to present the closed-form expressions of the
learning equations, that we particularize for some special
excitations. Our main goal is to discuss the features of the
neuron-model in an analytical way, in order to gain a deeper insight
into the behavior of the equations governing information-theoretic
non-linear unit learning.

Download from: http://www.unipg.it/~sfr/publications/NeCo2002.zip [8
pages, 576 KB]

Information-Theoretic Learning for FAN Network Applied to Eterokurtic 
Component Analysis
=============================================================================
Author: S. Fiori, Faculty of Engineering, University of Perugia (Italy)
Journal: IEE Proceedings -- Image, Vision, and Signal Processing

Extended abstract:

In this paper we deal with instantaneous linear mixtures and focus on
the stream of INFOMAX learning algorithms. The paper is devoted to the
separation of mixed independent signals from their linear mixtures
when the observations are mixed plati-kurtic and lepto-kurtic signals,
that is referred to as hybrid or eterokurtic sources problem.

We propose the use of networks formed by unsupervised adaptive
activation function neurons (FAN), which provide a natural way of
estimating the high-order statistical features required to achieve
separation.

Through numerical and analytical studies the effectiveness of the
presented approach is also illustrated and discussed. In Section 2 the
problem at hand is formally presented and the adaptive activation
function structure is shown to emerge as a natural solution. In
Section 3 the general unsupervised learning theory for the FAN neuron
is derived, along with a closely-related one, based on a
mixture-of-kernel architecture, which is considered for further
numerical and architectural comparisons.  In Section 4 four different
FAN structures are proposed and discussed, while Section 5 is devoted
to computer simulations and comparisons.

Download from: http://www.unipg.it/~sfr/publications/EKA2002.zip [27
pages, 483 KB]

===================================================
Dr Simone Fiori (Mr, EE, PhD)- Assistant Professor
Faculty of Engineering - Perugia University
Via Pentima bassa, 21 - 05100 TERNI (Italy)
eMail: sfr at unipg.it - Fax: +39 0744 492925
Web: http://www.unipg.it/~sfr/
===================================================