Connectionists: Ongoing Neural Network ReNNaissance

[Juergen Schmidhuber]

A special breed of fast deep neural networks keeps winning important  
pattern recognition competitions, lately even with human-competitive  
results.

New greatly improved world records on visual pattern recognition  
benchmarks:

• 3. MNIST Handwritten Digit Recognition (perhaps the most famous  
machine learning benchmark). The new record is 0.23% error rate. This  
is the first human-competitive result on MNIST. The best result by  
others (brbo) is 0.39%.
• 2. NORB Object Recognition Benchmark. New record on full NORB: 2.7%  
(brbo 5%)
• 1. CIFAR-10 Object Recognition Benchmark. New record 11.2% (brbo  
18.5%)

Upcoming paper on this with Dan Cireşan & Ueli Meier at CVPR 2012:
Multi-Column Deep Neural Networks for Image Classification
(long preprint available)

1st ranks in visual pattern recognition competitions:

• 7. March 2012: ISBI 2012 Segmentation Challenge, won on all three  
evaluation metrics by a large margin, with superhuman pixel error rate.
• 6. August 2011: IJCNN 2011 on-site Traffic Sign Recognition  
Competition: 0.56% error rate - nearly three times better than brbo.  
The only method outperforming humans.
• 5. June 2011: ICDAR 2011 offline Chinese Handwriting Recognition  
Competition
• 4. January 2011: Online German Traffic Sign Recognition Contest  
(1st & 2nd rank)
• 3. ICDAR 2009 Arabic Connected Handwriting Competition, like the  
others below won by LSTM recurrent neural networks (deep by nature).
• 2. ICDAR 2009 Handwritten Farsi/Arabic Character Recognition  
Competition
• 1. ICDAR 2009 French Connected Handwriting Competition

No unsupervised pre-training!

Overview web site with more details and Google Tech Talk and  
contributions to neural net vision applications by authors including  
Dan Cireşan, Ueli Meier, Jonathan Masci, Alessandro Giusti, Alex  
Graves, Jawad Nagi, Frederic Ducatelle, Gianni Di Caro, Luca Maria  
Gambardella:

http://www.idsia.ch/~juergen/vision.html


Jürgen Schmidhuber
Director of the Swiss AI Lab IDSIA, Lugano
Professor of Artificial Intelligence, Univ. Lugano
Professor SUPSI, Manno-Lugano, Switzerland
http://www.idsia.ch/~juergen/whatsnew.html