Connectionists: Scientific Integrity, the 2021 Turing Lecture, etc.

[Stephen José Hanson]

Juergen:

'And even today, many students are taught NNs like this: let's start with a linear single-layer NN (activation = sum of weighted inputs). Now minimize mean squared error on the training set. That's good old linear regression (method of least squares"

Indeed there are many thing students are taught that are wrong or misleading for simplification or from ignorance..  That doesn't justify more of them.

So this is multiple linear regression you are talking about it..  but, again  a different model from a Neural Network.

Not a matter of math.. not talking eigenvectors here, we are still talking about a model of a biological neuron.   Maybe not a great model, but a step in the direction of brain like modeling.   Multiple regression is not such a model.

minimizing sums of squares and taking partials wrt parameters will result in formulae for beta weights and intercepts.  A useful model for interpreting the linear effects of non-collinear variables on a response.   widely useful in many scientific fields.    But not a Neural Network--not a model of neurons and synapses and dendrites.  Nonetheless, a useful pragmatic model developed for matrices of data with multiple variables and observations.

There was simply no reason that Frank Rosenblatt should have referenced this math, as it had nothing whatsoever to do with the Perceptron, since no partials of sums of squares could be computed.   Its the math and should be clear now.

Steve

On 1/2/22 8:43 AM, Schmidhuber Juergen wrote:
> And even today, many students are taught NNs like this: let's start with a linear single-layer NN (activation = sum of weighted inputs). Now minimize mean squared error on the training set. That's good old linear regression (method of least squares
-- 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220102/fa63f3d5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.png
Type: image/png
Size: 19957 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220102/fa63f3d5/attachment.png>