Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

[Schmidhuber Juergen]

Thanks, Barak! Indeed, I should have said in the email msg that _others_ interpreted the book of Minsky & Papert [M69] in this way. My report explicitly mentions Terry [S20] who wrote in 2020:

"The great expectations in the press (Fig. 3) were dashed by Minsky and Papert (7), who showed in their book Perceptrons that a perceptron can only represent categories that are linearly separable in weight space. Although at the end of their book Minsky and Papert considered the prospect of generalizing single- to multiple-layer perceptrons, one layer feeding into the next, they doubted there would ever be a way to train these more powerful multilayer perceptrons. Unfortunately, many took this doubt to be definitive, and the field was abandoned until a new generation of neural network researchers took a fresh look at the problem in the 1980s.” 

However, as mentioned above, the 1969 book [M69] addressed a "problem" of Gauss & Legendre's shallow learning (~1800) [DL1-2] that had already been solved 4 years prior by Ivakhnenko & Lapa's popular deep learning method [DEEP1-2][DL2] (and then also in 1967 by Amari's SGD for MLPs [GD1-2]). Deep learning research was not abandoned in the 1970s. It was alive and kicking, especially outside of the Anglosphere. [DEEP2][GD1-3][CNN1][DL1-2]

See Sec. II and Sec. XIII of the report: https://people.idsia.ch/~juergen/scientific-integrity-turing-award-deep-learning.html 

Cheers,
Jürgen




> On 1 Feb 2022, at 14:17, Barak A. Pearlmutter <barak at pearlmutter.net> wrote:
> 
> Jürgen,
> 
> It's fantastic that you're helping expose people to some important bits of scientific literature.
> 
> But...
> 
> > Minsky & Papert [M69] made some people think that Rosenblatt [R58-62] had only linear NNs plus threshold functions
> 
> If you actually read Minsk and Papert's "Perceptrons" book, this is not a misconception it encourages. It defines a "k-th order perceptron" as a linear threshold unit preceded by an arbitrary set of fixed nonlinearities with fan-in k. (A linear threshold unit with binary inputs would, in this terminology, be a 1st-order perceptron.) All their theorems are for k>1. For instance, they prove that a k-th order perceptron cannot do (k+1)-bit parity, which in the special case of k=1 simplifies to the trivial observation that a simple linear threshold unit cannot do xor.
> <perceptrons-book-cover-1.jpg> <perceptron-diagram-1.jpg>
> This is why you're not supposed to directly cite things you have not actually read: it's too easy to misconstrue them based on inaccurate summaries transmitted over a series of biased noisy compressive channels.
> 
> Cheers,
> 
> --Barak.