No subject

[Stephen J Hanson]

     Princeton Cognitive Science Lab Technical Report: CSL36, February, 1989.

                         COMPARING BIASES FOR MINIMAL NETWORK
                          CONSTRUCTION WITH BACK-PROPAGATION


                                 Stephen Jos'e Hanson

                                       Bellcore
                                          and
                        Princeton Cognitive Science Laboratory

                                         and

                                    Lorien Y. Pratt
                                  Rutgers University


                                       ABSTRACT

         Rumelhart (1987), has proposed a method for choosing minimal or
         "simple" representations during learning in Back-propagation
         networks.  This approach can be used to (a) dynamically select
         the number of hidden units, (b) construct a representation that
         is appropriate for the problem and (c) thus improve the
         generalization ability of Back-propagation networks. The method
         Rumelhart suggests involves adding penalty terms to the usual
         error function. In this paper we introduce Rumelhart's minimal
         networks idea and compare two possible biases on the weight
         search space.  These biases are compared in both simple counting
         problems and a speech recognition problem.  In general, the
         constrained search does seem to minimize the number of hidden
         units required with an expected increase in local minima.

to appear in Advances in Neural Information Processing, D. Touretzky Ed., 1989
Research was jointly sponsered by Princeton CSL and Bellcore.


REQUESTS FOR THIS TECHNICAL REPORT SHOULD BE SENT TO

    laura at clarity.princeton.edu


  Please do not reply to this message or forward, Thankyou.