New Paper: infomax and redundancy reduction

[NADAL Jean-Pierre]

FTP-host: archive.cis.ohio-state.edu
FTP-file: pub/neuroprose/nadalparga.infomaxredred.ps.Z

The following paper has been placed in
the Neuroprose repository:

"Non linear neurons in the low noise limit:
a factorial code maximizes information transfer"
J.-P. Nadal (ENS, Paris) and N. Parga (UAM, Madrid)

(19 pages)
To appear in NETWORK: computation in neural systems

ABSTRACT:
We investigate the consequences of maximizing information transfer
in a simple neural network (one input layer, one output layer),
focussing on the case of {\it non linear} transfer
functions. We assume that both receptive fields
(synaptic efficacies) and transfer functions can be
adapted to the environment.
he main result is that, for bounded and invertible transfer functions,
in the case of a vanishing {\it additive} output noise, and no
input noise, maximization of information
(Linsker's{\it infomax} principle)
leads to a factorial code - hence to the same solution as
required by the redundancy reduction principle of Barlow.

We show also that this result is valid for {\it linear},
more generally unbounded,
transfer functions, provided optimization is performed under an
additive constraint, that is which can be written as a
sum of terms, each one being specific to one output neuron.

Finally we study the effect of a non zero input noise. We find that,
at first order in the input noise, assumed to be small as compared
to the - small - output noise,
the above results are still valid, provided the {\it output} noise
is uncorrelated from one neuron to the other.

Jean-Pierre Nadal
nadal at physique.ens.fr
Laboratoire de Physique Statistique
Ecole Normale Sup\'erieure
24, rue Lhomond
75231 Paris Cedex05 - France

Nestor Parga
parga at ccuam3.sdi.uam.es
Departamento de F\'isica Te\'orica
Universidad Aut\'onoma de Madrid
Canto Blanco, 28049 Madrid - Spain
 
============================================
 
Retrieve this paper by anonymous ftp from:
archive.cis.ohio-state.edu
in the /pub/neuroprose directory
 
The name of the paper in this archive is:
nadalparga.infomaxredred.ps.Z
(19 pages. No hard copy available.)