Connectionists: Who introduced the term "Deep Learning" to NNs?

[Geoffrey Hinton]

I think the current popularity of the term started with the paper by
Hinton Osindero and Teh in 2006 called "A fast learning algorithm for
deep belief nets".  After this paper there was a lot of talk about
deep belief nets.  In about 2007 the term "deep belief net" started
changing its meaning and was used (rather sloppily) to refer to deep
neural nets that were pre-trained as deep belief nets. The term gained
a lot of popularity because these nets were used to make good acoustic
models and that triggered the re-introduction of neural nets into
mainline speech recognizers. People eventually made a clear
terminological distinction between deep belief nets (DBNs) and deep
neural nets that were initialized as deep belief nets (DNNs or
DBN-DNNs). Then they discovered that with large datasets and sensible
initial scales for the weights the pre-training was not needed and
they generalized DNNs to any old deep neural net.

Its clearly true that people had previously used the term deep neural
net but that was not the origin of the resurgence of the term in about
2007.

Its pretty obvious by now that deep neural networks of the type that
people were using in the 1980's work very well when they have enough
data and enough computation, and its pretty obvious that the deep
convnets that Yann has been using since about 1987 are deep neural
nets, so what does it matter where the name came from?  Deep neural
nets are finally living up to their promise so lets all enjoy it.

Geoff




On Thu, Mar 12, 2015 at 1:58 PM, Schmidhuber Juergen <juergen at idsia.ch> wrote:
> Dear connectionists,
>
> to my knowledge, the ancient term "Deep Learning" was introduced to the NN field by Aizenberg & Aizenberg & Vandewalle's book (2000): "Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications."
>
> Is anyone aware of older NN papers using it?
>
> (Of course, the field itself is much older - Ivakhnenko started his work on deep learning networks in the mid 1960s.)
>
> Thanks!
>
> Juergen
>
> http://people.idsia.ch/~juergen/whatsnew.html