Connectionists: Who introduced the term "Deep Learning" to NNs?

[Leslie Smith]

Dear all:

Though it doesn’t quite go back to the fishermen of Northern Spain, it’s worth noting that multi-layer (and hence deep) nets are discussed in some detail by Rosenblatt in his “Principles of Neurodynamics” (1962 Spartan Books), specifically section 15, page 313 et seq, 

Clearly, he did not use the term “deep learning”: he talks about “adaptive pre terminal networks” when referring to alterations of weights in earlier layers. 

—Leslie Smith

On 13 Mar 2015, at 16:53, Schmidhuber Juergen <juergen at idsia.ch> wrote:

> Sorry, but the “semantics of what researchers nowadays call deep learning" are much older. In RNNs, the deepest of all NNs, your "strictly unsupervised followed by supervised finetuning” goes back to Schmidhuber's hierarchical deep RNN stacks of 1991 (the neural history compressors). They were largely replaced (still in the 1990s) by deep supervised LSTM RNNs. History repeated itself between 2006 and 2010, when deep unsupervised FNN stacks (kudos to Hinton et al) were replaced by deep standard supervised FNNs, as you pointed out. (It's hardly clear, however, that the re-popularization of supervised NNs wouldn't have occurred without the work on unsupervised NNs.)
> 
> Antoine Bordes' Google-generated graph seems to indicate that the usage of the term went up right after Aizenberg et al.’s book came out (2000). As Yoshua Bengio pointed out, however, it includes all kinds of ancient usages of “Deep Learning,” and is not limited to NN-specific usage in the sense of this thread. 
> 
> Again, I am just trying locate the introduction of the term. It's an interesting question in its own right, outside of when the principles of deep learning came into being. 
> 
> Juergen 

Leslie Smith
l.s.smith at cs.stir.ac.uk
Professor of Computing,
Computing Science and Mathematics,
University of Stirling
Stirling FK9 4LA,
Scotland, UK
+44 (0) 1786 467435





-- 
The University of Stirling has been ranked in the top 12 of UK universities for graduate employment*.
94% of our 2012 graduates were in work and/or further study within six months of graduation.
*The Telegraph
The University of Stirling is a charity registered in Scotland, number SC 011159.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20150314/b16b63f2/attachment.html>