neural network success story

[Chris Burges]

> b) What are the "big success stories" (i.e., of the kind the general public
> could understand) for neural networks contributing to the construction of
> "artificial" brains, i.e., successfully fielded applications of NN hardware
> and software that have had a major commercial or other impact?

Lucent Technologies sells the Lucent Courtesy Amount Reader (LCAR) to read
financial amounts on US checks.  This software is currently installed in a
number of banks and is processing several million checks per day.  LCAR reads
machine print and handwritten amounts on both personal and business checks.  The
amount recognition algorithms are based on feed-forward convolutional neural
networks.

The basic ideas underlying the graph-based approach to both segmentation and
neural network training can be found in:

C.J.C. Burges, O. Matan, Y. Le Cun, J.S. Denker, L.D. Jackel, C.E. Stenard,
C.R. Nohl, J.I. Ben, "Shortest Path Segmentation: A Method For Training a Neural
Network to Recognize Character Strings", IJCNN Conference Proceedings Vol 3,
pp. 165-172, 1992

J. Denker, C.J.C. Burges, "Image Segmentation and Recognition", in The
Mathematics of Generalization: Proceedings of the SFI/CNLS Workshop on Formal
Approaches to Supervised Learning, Addison Wesley, ISBN 0-201-40985-2, 1994

C.J.C. Burges, J.I. Ben, J.S. Denker, Y. LeCun and C.R. Nohl, "Off Line
Recognition of Handwritten Postal Words Using Neural Networks", International
Journal of Pattern Recognition and Artificial Intelligence, Vol. 7, Number 4,
p. 689, 1993; also in Advances in Pattern Recognition Systems Using Neural
Network Technologies, Series in Machine Perception and Artificial Intelligence,
Volume 7, Edited by I. Guyon and P.S.P Wang, World Scientific, 1993.

More recently, the graph-based approach has been significantly extended to allow
end-to-end training of large, complex systems.  For this see:

Leon Bottou, Yoshua Bengio, Yann Le Cun, "Global Training of Document Processing
Systems using Graph Transformer Networks", In Proceedings of Computer Vision and
Pattern Recognition, Puerto Rico, IEEE, 1997.

An extended paper discussing this will appear soon in Transactions of IEEE:

Yann Le Cun, Leon Bottou, Yoshua Bengio, and Patrick Haffner, "Gradient Based
Learning Applied to Document Recognition", to appear in Proceedings of IEEE.

The underlying neural networks used by the system are the convolutional feed
forward "LeNet" series.  These are pretty well known by now.  One place to go
for a description, and a comparison with other algorithms, is:

Yann Le Cun, Lawrence D. Jackel, Leon Bottou, Corinna Cortes, John S. Denker,
Harris Drucker, Isabelle Guyon, Urs A. Muller, Eduard Sackinger, Patrice
Simard, and Vladimir N. Vapnik. Learning algorithms for classification: A
comparison on handwritten digit recognition. In J. H. Oh, C. Kwon, and S. Cho,
editors, Neural Networks: The Statistical Mechanics Perspective, pages 261-276.
World Scientific, 1995.

There is quite a bit more to the LCAR system than is represented by these
refs. (e.g. how to read handwritten fractional amounts), but those methods are
not yet written up anywhere.  However you can find a little more information on
the LCAR system itself at http://www.lucent.dk/ssg/html/lcar.html.


- Chris Burges
burges at lucent.com