NIPS workshop - REAL biological computation

[Jim Schwaber]

-----------NIPS 92 WORKSHOP----------------------

Real Applications of Real Biological Circuits

		    or

"If back-prop is not enough how will we get more?"

                    or

"Is anybody really getting anywhere with biology?"

---------------------------------------------------

When:   Friday, Dec. 4th
====

Intended Audience:   Those interested in detailed biological modeling.
==================    Those interested in nonlinear control.
		    Those interested in neuronal signal processing.
			Those interested in connecting the above.
	

Organizers:
===========
Richard Granger           Jim Schwaber
granger at ics.uci.edu       schwaber at eplrx7.es.dupont.com

Agenda:
=======  

Morning Session, 7:30 - 9:30, Brain Control Systems and Chemical
---------------				Process Control

Jim Schwaber		Brainstem reflexes as adaptive controllers
Dupont

Babatunde Ogunnaike 	Reverse engineering brain control systems
DuPont

Frank Doyle		Neurons as nonlinear systems for control
Purdue

John Hopfield   	Discussant
Caltech

Afternoon Session, 4:30 - 6:30, Real biological modeling, nonlinear
-----------------		systems and signal processing

Richard Granger         Signal processing in real neural systems:  is
UC Irvine		it applicable?
			
Gary Green              The single neuron as a nonlinear system - its
Newcastle		Volterra kernels as described by neural networks.


Program: 
========

We anticipate that the topic will generate several points of view.
Thus, presenters will restrict themselves to a very, very few slides
intended to make a point for discussion.  Given that there now are
concrete examples of taking biological principles to application, we
expect the discussion will center more on how, and at what level,
rather than whether "reverse engineering the brain" is useful.

Granger (UC Irvine):
-------
The architectures, performance rules and learning rules of most artificial
neural networks are at odds with the anatomy and physiology of real
biological neural circuitry.  For example, mammalian telencephelon
(forebrain) is characterized by extremely sparse connectivity (~1-5%),
almost entirely lacks dense recurrent connections, and has extensive lateral
local circuit connections; inhibition is delayed-onset and relatively
long-lasting (100s of milliseconds) compared to rapid-onset brief excitation
(10s of milliseconds), and they are not interchangeable.  Excitatory
connections learn, but there is very little evidence for plasticity in
inhibitory connections.  Real synaptic plasticity rules are sensitive to
temporal information, are not Hebbian, and do not contain "supervision"
signals in any form related to those common in ANNs.

These discrepancies between natural and artificial NNs raise the question of
whether such biological details are largely extraneous to the behavioral and
computational utility of neural circuitry, or whether such properties may
yield novel rules that confer useful computational abilities to networks
that use them.  In this workshop we will explicitly analyze the power and
utility of a range of novel algorithms derived from detailed biology, and
illustrate specific industrial applicatons of these algorithms in the fields
of process control and signal processing.

Ogunnaike (DuPont):
-----------
REVERSE ENGINEERING BRAIN CONTROL SYSTEMS:
EXPLORING THE POTENTIAL FOR APPLICATIONS IN CHEMICAL PROCESS CONTROL.
=====================================================================

The main motivation for our efforts lies in the simple fact that there
are remarkable analogies between the human body and the chemical
process plant.  Furthermore, it is known that while the brain has been
quite successful in performing its task as the central supervisor of
intricate control systems operating under conditions which leave very
little margin for error, the control computer in the chemical process
plant has not been so successful.


We have been concerned with seeking answers to the following question:  

``Is it possible to ``reverse engineer'' a biological control system
and use the understanding to develop novel approaches to chemical
process control systems design and analysis?''

Our discussion will provide an overview of the tentative answers we
have to date.  We will first provide a brief summary of the salient
features and main problems of chemical process control; we will then
introduce the biological control system under study (the baroreceptor
vagal reflex); finally we will present an actual industrial process
whose main features indicate that it may benefit from the knowledge
garnered from the neurobiological studies.

Doyle (Purdue):
------
We are focusing our research on two levels:
1) Neuron level: investigating novel building blocks for process
modeling applications which are motivated by realistic biological
neurons.
2) Network Level: looking for novel approaches to nonlinear dynamic
scheduling algorithms for process control and modeling (again,
motivated by biological signal processing in the baroreceptor reflex).

Green (Newcastle):
-------
I would love to tell the NIPS people about Volterra series,
especially as we have now made a connection between neural
networks, Volterra series and the differential geometric
representation of networks. This allows us to say why one, two or
more layers are necessary for a particular analytic problem. We can
also say how to invert nets which are homeomorphic in their
mappings. More importantly for us biologists we can turn the state
equations of membrane currents, using neural networks into
approximate Volterra kernels which I think (!) helps understand the
dynamics. This gives a solution to the differential equations,
albeit an approximate one in practical terms. The equations are
invertible and therefore allow a formal link between current clamp
and voltage clamp at the equation level. The method we have used to
do this is of interest to chem. eng. people because we can use the
same concepts in non-linear control. It appears at first glance
that we can link the everyday use of neural networks to well
established theory through a study of tangent spaces of networks.
We construct a state space model of a plant, calculate the
differential of the rate of change of output with respect to the
input. Calculate the same for a neural network. Compare
coefficients. The solution to the set of simultaneous equations for
the coefficents produces a network which is formally equivalent to
the solution of the original differential equation which defined
the state equations. 


We will be making the claim that analytic solutions of non-linear
differential equations is possible using neural networks for
some problems. For all other problems an approximate solution is
possible but the architecture that must be used can be defined.
Last I'll show how this is related to the old techniques using
Volterra series and why the kernels and inverse transforms can be
directly extracted from networks. I think it is a new method of
solving what is a very old problem. All in 20 minutes !