Connectionists: Early history of symbolic and neural network approaches to AI

[Gary Marcus]

- certainly in broad agreement; never argued that symbols alone were sufficient.
- have always since my first paper in 1992 advocated for hybrid models, with both associative and symbolic-components
- as argued in my 2001 book, i think the real question is not whether we need symbols per se (which localist output units represent) but rather AI/models of human cognition require algebraic operations over variables in order to be robust, in addition to other mechanisms. 

> On Feb 18, 2024, at 09:25, poole <poole at cs.ubc.ca> wrote:
> 
> Thanks Gary.
> These are both worthwhile reading!
> 
> I don’t think symbolic = logic.  McCulloch and Pitts were interested in representing logical operations.
> 
> “Symbolic" follows the tradition of Hobbes (1588–1679) who claimed that thinking was symbolic reasoning, like talking out loud or working out an answer with pen and paper [see Haugeland, J. Artificial Intelligence: The Very Idea. MIT Press  1985].  Newell and Simon [1976] defined a symbol is a meaningful pattern that can be manipulated. A symbol system creates, copies, modifies, and destroys symbols.
> 
> Graphical models and believe networks typically have symbolic random variables.
> 
> It is very common for modern neural networks to have symbolic inputs or outputs, e.g., words, knowledge graphs, molecular structure, game moves,…
> 
> I don’t think Gary would disagree that there needs to be some non-symbols (e.g, hidden units in neural networks).
> 
> Arguments for symbols — the most compelling one for me is that organizations (which are much more intelligent than individuals) reason in terms of symbols (words, diagrams, spreadsheets) — are not diminished by the need for non-symbols.
> 
> David
> (references from artint.info where many of these issues are discussed).
> 
>> On Feb 17, 2024, at 5:42 PM, Gary Marcus <gary.marcus at nyu.edu> wrote:
>> 
>> [CAUTION: Non-UBC Email]adding some pointers to David’s remarks: McCulloch and Pitts in 1943 were very much trying to bridge the symbolic and neural world. It’s clear even in the abstract (article below). Tensions between symbolic and neural approaches were in full force by Minsky and Papert 1969, and resurfaced in the 1980s.
>> 
>> I don’t have a clear sense of where things were in Turing’s time per se, but both approaches were countenanced in the 1955 proposal for the Dartmouth conference link below; Rosenblatt had gathered steam by 1958 as noted.
>> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__web.archive.org_web_20070826230310_http-3A__www-2Dformal.stanford.edu_jmc_history_dartmouth_dartmouth.html&d=DwIFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=i1FPhi6c93rU7h_AikZJEx36pDsvgNzWMEKlOMHY0uqMdsAuRf5w_wtT_l9EhRaQ&s=qB3xwLa8UE9Dg_o7t21jk3miiMP1icWlWBL_XGQ49-Q&e=
>> 
>> 
>> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__home.csulb.edu_-7Ecwallis_382_readings_482_mccolloch.logical.calculus.ideas.1943.pdf&d=DwIFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=i1FPhi6c93rU7h_AikZJEx36pDsvgNzWMEKlOMHY0uqMdsAuRf5w_wtT_l9EhRaQ&s=6jCRjchELvEA_Dzv8hAwzcYSMdGNQiQNU_z3DO6ddp4&e=
>> 
>> 
>> 
>>>> On Feb 17, 2024, at 4:40 PM, poole <poole at cs.ubc.ca> wrote:
>>> 
>>> 
>>>> On Feb 17, 2024, at 1:08 PM, David H Kirshner <dkirsh at lsu.edu> wrote:
>>>> 
>>>> [CAUTION: Non-UBC Email]
>>>> You’re right, David.
>>>> I should have said “Back in Alan Turing’s time when the possibility of AI meant the possibility of symbolic AI, ….”
>>> 
>>> In Turing's time, from what I can see (I wan’t alive then ;^) neural networks were more trendy than symbolic approaches. Turing’s paper was 1950. McCulloch and Pitts seminal work was 1943. Minsky’s thesis on neural networks was written in 1952. (Schmidhuber has great resources on the history of NNs and AI on his website).
>>> 
>>> There was lots of neural network hype in the 1950’s:
>>> 
>>> "The Navy revealed the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence. …The service said it would …build the first of its Perceptron thinking machines that will be able to read and write. It is expected to be finished in about a year at a cost of $100,000."
>>> – New York Times [1958]
>>> 
>>> It was later in the 1950’s that they came to realize that AI needed representations, lead by Minsky and McCarthy, whick lead to the rise of symbolic approaches.. (It is interesting that a major NN conference ICLR is about representations).
>>> 
>>> I am sure there are people who know the history better than me, who might like to provide more persoective.
>>> 
>>> David
>>> 
>>> 
>>>> ——
>>>> David Poole,  
>>>> Department of Computer Science,   
>>>> University of British Columbia,   
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__cs.ubc.ca_-7Epoole&d=DwIFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=wQR1NePCSj6dOGDD0r6B5Kn1fcNaTMg7tARe7TdEDqQ&m=GaLFSysAmNesHoiuLMLNQzFZKcQfTn2lkEhxM8Xrc205FjMs-5qx1lSBZ4u9kagl&s=XFNzK3B_BUrmI0gbNaacNriRQ53tTCzjlNaL2JBAmRg&e=
>>>> poole at cs.ubc.ca
>>> 
>>> 
>