2 papers in NEUROPROSE on spiking versus sigmoidal neurons

[Wolfgang Maass]

1)
The file maass.third-generation.ps.Z is now available for copying
from the Neuroprose repository. This is a 23-page long paper.
Hardcopies are not available.

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/maass.third-generation.ps.Z


                 Networks of Spiking Neurons: 
         The Third Generation of Neural Network Models
        
                            Wolfgang Maass

              Institute for Theoretical Computer Science
                     Technische Universitaet Graz
                         Klosterwiesgasse 32/2
                         A-8010 Graz, Austria
                    e-mail: maass at igi.tu-graz.ac.at


                            Abstract

The computational power of formal models for networks of spiking 
neurons is compared with that of traditional neural network models
based on McCulloch Pitts neurons (i.e. threshold gates)
respectively sigmoidal gates. It is shown
that networks of spiking neurons are computationally more
powerful than threshold circuits and sigmoidal neural nets of the 
same size. 
A concrete biologically relevant function is exhibited which can be
computed by a single spiking neuron (for biologically
reasonable values of its parameters), but which requires
hundreds of hidden units on a sigmoidal neural net.

This article does not assume prior knowledge about spiking neurons,
and it  contains an extensive list of references
to the currently available literature on computations
in networks of spiking neurons and relevant results 
from neurobiology. 


***************************************************************
2)
The file maass.sigmoidal-spiking.ps.Z  is now also available for copying
from the Neuroprose repository. This is a 27-page long paper.
Hardcopies are not available.

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/maass.sigmoidal-spiking.ps.Z

   An Efficient Implementation of Sigmoidal Neural Nets in Temporal
               Coding with Noisy Spiking Neurons
   
                            Wolfgang Maass

              Institute for Theoretical Computer Science
                     Technische Universitaet Graz
                         Klosterwiesgasse 32/2
                         A-8010 Graz, Austria
                    e-mail: maass at igi.tu-graz.ac.at


                               Abstract

We show that networks of spiking neurons can simulate arbitrary 
feedforward sigmoidal neural nets in a way which has previously
not been considered.
This new approach is based on temporal coding by single spikes 
(respectively by the timing of synchronous firing in pools of neurons), 
rather than on the traditional interpretation of analog variables in
terms of firing rates. It is based on the observation that incoming
"postsynaptic potentials" can SHIFT the firing time of a spiking
neuron. The resulting new simulation is substantially faster and 
hence more consistent with experimental results about the 
speed of information processing in cortical neural systems.

As a consequence we can show that networks of noisy spiking neurons are
                     "universal approximators" 
in the sense that they can approximate with regard to
temporal coding any given continuous function of several variables.
This result holds for a fairly large class of schemes for coding analog
variables by firing times of spiking neurons. 

Our new proposal for the possible organization of computations
in networks of spiking neurons systems has some interesting 
consequences for the type of learning rules that would be needed to
explain the self-organization of such networks.

Finally, our fast and noise-robust implementation of sigmoidal neural
nets via temporal coding points to possible new ways of implementing
feedforward and recurrent sigmoidal neural nets with pulse stream VLSI.

(To appear in Neural Computation.)