techreport on application of AdaBoost to neural networks

[Holger Schwenk]

Hello,

The following technical report on the application of AdaBoost to neural
networks is available on the WWW page:

   http://www.iro.umontreal.ca/~lisa/pointeurs/AdaBoostTR.ps
or http://www.iro.umontreal.ca/~schwenk/papers/AdaBoostTR.ps.gz

Comments and suggestions are welcome.

 Holger Schwenk

-------------------------------------------------------------------------------
Holger Schwenk                                 phone: (514) 343-6111 ext 1655
                                                 fax: (514) 343-5834     
LISA, Dept. IRO                            
University of Montreal                         email: schwenk at iro.umontreal.ca
2920 Chemin de la tour, CP 6128           http://www.iro.umontreal.ca/~schwenk
Montreal, Quebec, H3C 3J7
CANADA
-------------------------------------------------------------------------------


       Adaptive Boosting of Neural Networks for Character Recognition

                      Holger Schwenk and Yoshua Bengio 

               Dept. Informatique et Recherche Operationnelle
            Universite de Montreal, Montreal, Qc H3C-3J7, Canada

                     {schwenk,bengioy}@iro.umontreal.ca

                              May, 29 1997



    "Boosting" is a general method for improving the performance of any
    learning algorithm that consistently generates classifiers which need
    to perform only slightly better than random guessing. A recently proposed
    and very promising boosting algorithm is AdaBoost [5]. It has been applied
    with great success to several benchmark machine learning problems using
    rather simple learning algorithms [4], in particular decision trees [1,2,6].
    In this paper we use AdaBoost to improve the performances of neural
    networks applied to character recognition tasks. We compare training
    methods based on sampling the training set and weighting the cost function.
    Our system achieves about 1.4% error on a data base of online handwritten
    digits from more than 200 writers. Adaptive boosting of a multi-layer
    network achieved 2% error on the UCI Letters offline characters data set.