Connectionists: Who introduced the term "Deep Learning" to NNs?

[Juergen Schmidhuber]

Thanks, Leslie! 

The title of this thread is about syntax, but everybody wants to talk about semantics … 

You mention Rosenblatt (1962), who heavily refers to Joseph’s even earlier work. His pre-terminal connections in section 15 use a simple type of unsupervised learning. 

To my knowledge, however, the first working, general, deep learning algorithms for multilayer perceptrons are due to Ivakhnenko et al (1965-). Still in use even in the new millennium!

Lots of references to deep learning networks of the 1960s and 1970s can be found in the survey, especially in Sec. 5.3, 5.4, 5.5. (Shallow learning networks since 1795 are mentioned in Sec. 5.1.)

http://people.idsia.ch/~juergen/deep-learning-overview.html

The nets of Spanish fishers are deep, but they don’t learn, otherwise I’d have cited them :-) 

Juergen





> On 14 Mar 2015, at 11:07, Leslie Smith <l.s.smith at cs.stir.ac.uk> wrote:
> 
> Dear all:
> 
> Though it doesn’t quite go back to the fishermen of Northern Spain, it’s worth noting that multi-layer (and hence deep) nets are discussed in some detail by Rosenblatt in his “Principles of Neurodynamics” (1962 Spartan Books), specifically section 15, page 313 et seq, 
> 
> Clearly, he did not use the term “deep learning”: he talks about “adaptive pre terminal networks” when referring to alterations of weights in earlier layers. 
> 
> —Leslie Smith