TR--Reflective Neural Network Architecture

[Frank Smieja]

The following paper has been placed in the Neuroprose archive.

*******************************************************************
	REFLECTIVE MODULAR NEURAL NETWORK SYSTEMS
	
		F. J. Smieja and H. Muehlenbein 

German National Research Centre for Computer Science (GMD)
		Schlo{\ss} Birlinghoven, 
		  5205 St. Augustin 1, 
			Germany.

			ABSTRACT

Many of the current artificial neural network systems have serious
limitations, concerning accessibility, flexibility, scaling and
reliability. In order to go some way to removing these we suggest a
{\it reflective neural network architecture}. In such an architecture,
the modular structure is the most important element.  The
building-block elements are called ``\MINOS'' modules.  They perform
{\it self-observation\/} and inform on the current level of
development, or scope of expertise, within the module.  A {\it
Pandemonium\/} system integrates such submodules so that they work
together to handle mapping tasks.  Network complexity limitations are
attacked in this way with the Pandemonium problem decomposition
paradigm, and both static and dynamic unreliability of the whole
Pandemonium system is effectively eliminated through the generation
and interpretation of {\it confidence\/} and {\it ambiguity\/}
measures at every moment during the development of the system.

Two problem domains are used to test and demonstrate various aspects
of our architecture.  {\it Reliability\/} and {\it quality\/} measures
are defined for systems that only answer part of the time.  Our system
achieves better quality values than single networks of larger size for
a handwritten digit problem.  When both second and third best answers
are accepted, our system is left with only 5\% error on the test set,
2.1\% better than the best single net.  It is also shown how the
system can elegantly learn to handle garbage patterns.  With the
parity problem it is demonstrated how complexity of problems may be
decomposed automatically by the system, through solving it with
networks of size smaller than a single net is required to be.  Even
when the system does not find a solution to the parity problem,
because networks of too small a size are used, the reliability remains
around 99--100\%.

Our Pandemonium architecture gives more power and flexibility to the
higher levels of a large hybrid system than a single net system can,
offering useful information for higher-level feedback loops, through
which reliability of answers may be intelligently traded for less
reliable but important ``intuitional'' answers.  In providing weighted
alternatives and possible generalizations, this architecture gives the
best possible service to the larger system of which it will form part.

Keywords:  Reflective architecture, Pandemonium, task decomposition,
	   confidence, reliability. 
********************************************************************


----------------------------------------------------------------
FTP INSTRUCTIONS

     unix% ftp archive.cis.ohio-state.edu (or 128.146.8.52)
     Name: anonymous
     Password: neuron
     ftp> cd pub/neuroprose
     ftp> binary
     ftp> get smieja.reflect.ps.Z
     ftp> bye
     unix% zcat smieja.reflect.ps.Z | lpr
(or whatever *you* do to print a compressed PostScript file)
----------------------------------------------------------------


	-Frank Smieja