Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

[Barak A. Pearlmutter]

One point of scientific propriety and writing that may be getting lost
in the scrum here, and which has I think contributed substantially to
the somewhat woeful state of credit assignment in the field, is the
traditional idea of what a citation *means*.

If a paper says "we use the Foo Transform (Smith, 1995)" that,
traditionally, implies that the author has actually read Smith (1995)
and it describes the Foo Transform as used in the work being
presented. If the author was told that the Foo Transform was actually
discovered by Barker (1980) but the author hasn't actually verified
that by reading Barker (1980), then the author should NOT just cite
Barker. If the author heard that Barker (1980) is the "right" citation
for the Foo Transform, but they got the details of it that they're
actually using from Smith (1995) then they're supposed to say so: "We
use the Foo Transform as described in Smith (1995), attributed to
Barker (1980) by someone I met in line for the toilet at NeurIPS
2019".

This seemingly-antediluvian practice is to guard against people citing
"Barker (1980)" as saying something that it actually doesn't say,
proving a theorem that it doesn't, defining terms ("rate code", cough
cough) in a fashion that is not consistent with Barker's actual
definitions, etc. Iterated violations of this often manifest as
repeated and successive simplification of an idea, a so-called game of
telephone, until something not even true is sagely attributed to some
old publication that doesn't actually say it.

So if you want to cite, say, Seppo Linnainmaa for Reverse Mode
Automatic Differentiation, you need to have actually read it yourself.
Otherwise you need to do a bounce citation: "Linnainman (1982)
described by Schmidhuber (2021) as exhibiting a Fortran implementation
of Reverse Mode Automatic Differentiation" or something like that.

This is also why it's considered fine to simply cite a textbook or
survey paper: nobody could possibly mistake those as the original
source, but they may well be where the author actually got it from.

To bring this back to the present thread: I must confess that I have
not actually read many of the old references Jürgen brings up.
Certainly "X (1960) invented deep learning" is not enough to allow
someone to cite them. It's not even enough for a bounce citation. What
did they *actually* do? What is Jürgen saying they actually did?