Evolving Networks - New TR

[Rik Belew]

			  EVOLVING NETWORKS:
		     USING THE GENETIC ALGORITHM
		     WITH CONNECTIONIST LEARNING
				   
			   Richard K. Belew
			    John McInerney
			 Nicolaus Schraudolf
				   
	      Cognitive Computer Science Research Group
	       Computer Science & Engr. Dept. (C-014)
		    Univ. California at San Diego
			  La Jolla, CA 92093
			   rik at cs.ucsd.edu
				   
		    CSE Technical Report #CS90-174
			      June, 1990
				   
			       ABSTRACT

It is appealing to consider hybrids of neural-network learning
algorithms with evolutionary search procedures, simply because Nature
has so successfully done so.  In fact, computational models of
learning and evolution offer theoretical biology new tools for
addressing questions about Nature that have dogged that field since
Darwin.  The concern of this paper, however, is strictly
artificial: Can hybrids of connectionist learning algorithms and
genetic algorithms produce more efficient and effective algorithms
than either technique applied in isolation?  The paper begins with a
survey of recent work (by us and others) that combines Holland's
Genetic Algorithm (GA) with connectionist techniques and delineates
some of the basic design problems these hybrids share.  This analysis
suggests the dangers of overly literal representations of the network
on the genome (e.g., encoding each weight explicitly).  A preliminary
set of experiments that use the GA to find unusual but successful
values for BP parameters (learning rate, momentum) are also reported.
The focus of the report is a series of experiments that use the GA to
explore the space of initial weight values, from which two different
gradient techniques (conjugate gradient and back propagation) are then
allowed to optimize.  We find that use of the GA provides much greater
confidence in the face of the stochastic variation that can plague
gradient techniques, and can also allow training times to be reduced
by as much as two orders of magnitude.  Computational trade-offs
between BP and the GA are considered, including discussion of a
software facility that exploits the parallelism inherent in GA/BP
hybrids.  This evidence leads us to conclude that the GA's GLOBAL
SAMPLING characteristics compliment connectionist  LOCAL SEARCH
techniques well, leading to efficient and reliable hybrids.

	--------------------------------------------------

If possible, please obtain a postscript version of this technical report
from the pub/neuroprose directory at cheops.cis.ohio-state.edu.
Here are the directions:

/*** 	Note:  This file is not yet in place.  Give us a few days,	***/
/***	say until after 4th of July weekend, before you try to get it.	***/

unix> ftp cheops.cis.ohio-state.edu          # (or ftp 128.146.8.62)
Name (cheops.cis.ohio-state.edu:): anonymous
Password (cheops.cis.ohio-state.edu:anonymous): neuron
ftp> cd pub/neuroprose
ftp> type binary
ftp> get
(remote-file) evol-net.ps.Z
(local-file) foo.ps.Z
ftp> quit
unix> uncompress foo.ps.Z
unix> lpr -P(your_local_postscript_printer) foo.ps

/*** 	Note:  This file is not yet in place.  Give us a few days,	***/
/***	say until after 4th of July weekend, before you try to get it.	***/

If you do not have access to a postscript printer, copies of this
technical report can be obtained by sending requests to:

	Kathleen Hutcheson
	CSE Department (C-014)
	Univ. Calif. -- San Diego
	La Jolla, CA 92093

Ask for CSE Technical Report #CS90-174, and enclose $3.00 to cover
the cost of publication and postage.