Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

[Schmidhuber Juergen]

In a mature field like math we’d never have such a discussion. 

It is well-known that plagiarism may be unintentional (e.g., https://www.ox.ac.uk/students/academic/guidance/skills/plagiarism). However, since nobody can read the minds and intentions of others, science & tech came up with a _formal_ way of establishing priority: the time stamps of publications and patents. If you file your patent one day after your competitor, you are scooped, no matter whether you are the original inventor, a re-inventor, a great popularizer, or whatever. (In the present context, however, we are mostly talking about decades rather than days.)

Randy wrote: "For example, I cite Rumelhart et al (1986) for backprop, because that is how I and most other people in the modern field learned about this idea, and we know for a fact that they genuinely reinvented it and conveyed its implications in a very compelling way.  If I might be writing a paper on the history of backprop, or some comprehensive review, then yes it would be appropriate to cite older versions that had limited impact, being careful to characterize the relationship as one of reinvention."

See Sec. XVII of the report: "The deontology of science requires: If one `re-invents' something that was already known, and only becomes aware of it later, one must at least clarify it later, and correctly give credit in all follow-up papers and presentations."  

In particular, along the lines of Randy's remarks on historic surveys: from a survey like the 2021 Turing lecture you'd expect correct credit assignment instead of additional attempts at getting credit for work done by others. See Sec. 2 of the report: https://people.idsia.ch/~juergen/scientific-integrity-turing-award-deep-learning.html#lbhacm

I also agree with what Barak Pearlmutter wrote on the example of backpropagation: "So if you want to cite, say, Seppo Linnainmaa for Reverse Mode Automatic Differentiation, you need to have actually read it yourself. Otherwise you need to do a bounce citation: `Linnainmaa (1982) described by Schmidhuber (2021) as exhibiting a Fortran implementation of Reverse Mode Automatic Differentiation' or something like that.'" Indeed, you don't have to read Linnainmaa's original 1970 paper on what's now called backpropagation, you can read his 1976 journal paper (in English), or Griewank's 2012 paper "Who invented the reverse mode of differentiation?" in Documenta Mathematica, or other papers on this famous subject, some of them cited in my report, which also cites Werbos, who first applied the method to NNs in 1982 (but not yet in his 1974 thesis). The report continues: "By 1985, compute had become about 1,000 times cheaper than in 1970, and the first desktop computers had just become accessible in wealthier academic labs. Computational experiments then demonstrated that backpropagation can yield useful internal representations in hidden layers of NNs.[RUM] But this was essentially just an experimental analysis of a known method.[BP1-2] And the authors did not cite the prior art - not even in later surveys.[DL3,DL3a][DLC]"

I find it interesting what Asim Roy wrote: "In 1975 Tjalling Koopmans and Leonid Kantorovich were awarded the Nobel Prize in Economics for their contribution in resource allocation and linear programming. Many professionals, Koopmans and Kantorovich included, were surprised at Dantzig’s exclusion as an honoree. Most individuals familiar with the situation considered him to be just as worthy of the prize. [...] (Unbeknownst to Dantzig and most other operations researchers in the West, a similar method was derived eight years prior by Soviet mathematician Leonid V. Kantorovich)." Let's not forget, however, that there is no "Nobel Prize in Economics!" (See also this 2010 paper: https://people.idsia.ch/~juergen/nobelshare.html)

Jürgen