Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

[Schmidhuber Juergen]

Steve, almost all of deep learning is about engineering and problem solving, not about explaining or modeling biological neurons/synapses/dendrites. The most successful deep learners of today generalize the linear regressors of 1800. Same error. Same objective. Same basic architecture per unit or “neuron." Same weights per neuron. But now with nonlinear differentiable activation functions and deeper global architectures. Jürgen


> On 2 Jan 2022, at 17:55, Stephen José Hanson <jose at rubic.rutgers.edu> wrote:
> 
> Juergen:
> 
> 'And even today, many students are taught NNs like this: let's start with a linear single-layer NN (activation = sum of weighted inputs). Now minimize mean squared error on the training set. That's good old linear regression (method of least squares"
> 
> Indeed there are many thing students are taught that are wrong or misleading for simplification or from ignorance..  That doesn't justify more of them.
> 
> So this is multiple linear regression you are talking about it..  but, again  a different model from a Neural Network.
> 
> Not a matter of math.. not talking eigenvectors here, we are still talking about a model of a biological neuron.   Maybe not a great model, but a step in the direction of brain like modeling.   Multiple regression is not such a model.
> 
> minimizing sums of squares and taking partials wrt parameters will result in formulae for beta weights and intercepts.  A useful model for interpreting the linear effects of non-collinear variables on a response.   widely useful in many scientific fields.    But not a Neural Network--not a model of neurons and synapses and dendrites.  Nonetheless, a useful pragmatic model developed for matrices of data with multiple variables and observations.
> 
> There was simply no reason that Frank Rosenblatt should have referenced this math, as it had nothing whatsoever to do with the Perceptron, since no partials of sums of squares could be computed.   Its the math and should be clear now.
> 
> Steve
> 
> On 1/2/22 8:43 AM, Schmidhuber Juergen wrote:
>> And even today, many students are taught NNs like this: let's start with a linear single-layer NN (activation = sum of weighted inputs). Now minimize mean squared error on the training set. That's good old linear regression (method of least squares
> -- 
> <signature.png>