Two papers on information transfer / problem decomposition

[Lorien Y. Pratt]

The following two papers are now available via FTP from the neuroprose
archives.  The first is for AAAI91, so written towards an AI/Machine
learning audience.  The second is for IJCNN91, so more neural
network-oriented.  There is some overlap between them: the AAAI paper
reports briefly on the study describved in more detail in the IJCNN
paper.  Instructions for retrieval are at the end of this message.

	--Lori

#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@

	Direct Transfer of Learned Information Among Neural Networks

		       To appear: Proceedings of AAAI-91 


	      Lorien Y. Pratt and Jack Mostow and Candace A. Kamm

				   Abstract

      A touted advantage of symbolic representations is the ease of
      transferring learned information from one intelligent agent to
      another.  This paper investigates an analogous problem:  how to use
      information from one neural network to help a second network learn a
      related task.  Rather than translate such information into symbolic
      form (in which it may not be readily expressible), we investigate the
      direct transfer of information encoded as weights.

      Here, we focus on how transfer can be used to address the important
      problem of improving neural network learning speed.  First we present
      an exploratory study of the somewhat surprising effects of pre-setting
      network weights on subsequent learning.  Guided by hypotheses from this
      study, we sped up back-propagation learning for two speech recognition
      tasks.  By transferring weights from smaller networks trained on
      subtasks, we achieved speedups of up to an order of magnitude compared
      with training starting with random weights, even taking into account
      the time to train the smaller networks.  We include results on how
      transfer scales to a large phoneme recognition problem.

@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@

	      Improving a Phoneme Classification Neural Network  
		      through Problem Decomposition

		    To appear: Proceedings of IJCNN-91

			L. Y. Pratt and C. A. Kamm

				  Abstract

    In the study of neural networks, it is important to determine whether
    techniques that have been validated on smaller experimental tasks can
    be scaled to larger real-world problems.  In this paper we discuss how
    a methodology called {\em problem decomposition} can be applied to
    AP-net, a neural network for mapping acoustic spectra to phoneme
    classes.  The network's task is to recognize phonemes from a large
    corpus of multiple-speaker, continuously-spoken sentences.  We review
    previous AP-net systems and present results from a decomposition study
    in which smaller networks trained to recognize subsets of phonemes are
    combined into a larger network for the full signal-to-phoneme mapping
    task.  We show that, by using this problem decomposition methodology,
    comparable performance can be obtained in significantly fewer
    arithmetic operations.


^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%

To retrieve:

   	      unix> ftp cheops.cis.ohio-state.edu (or 128.146.8.62)
              Name: anonymous
              Password: neuron
              ftp> cd pub/neuroprose
              ftp> binary
              ftp> get pratt.aaai91.ps.Z
	      ftp> get pratt.ijcnn91.ps.Z
              ftp> quit
              unix> uncompress pratt.aaai91.ps.Z pratt.ijcnn91.ps.Z
              unix> lpr pratt.aaai91.ps pratt.ijcnn91.ps