Connectionists: Yann Le Cun and Stephen Hanson [EXTERNAL] Connectionists Digest, Vol 821, Issue 5
Sun, Ron
sunr at rpi.edu
Mon Dec 26 15:24:46 EST 2022
Dear Deokgun:
If you are looking for cognitive architectures that incorporate both System 1 and System 2 (i.e., dual-process theories), there have been some over the years. You may even trace it back to the 1990s, when such ideas first emerged (way before Daniel Kahnenman and a number of other researchers popularized this notion in the 2000s and 2010s).
In general, as a survey of this early history, I refer you to the following edited volume:
Sun, R. & Bookman, L. (1994). Computational Architectures Integrating Symbolic and Connectionist Processing<https://www.google.com/url?q=https%3A%2F%2Flink.springer.com%2Fbook%2F10.1007%2Fb102608&sa=D&sntz=1&usg=AFQjCNH0S8Keh-zd4GDNiSuOq6TnWdi2zw>. Kluwer Academic Publishers, The Netherlands.
For a more recent and better developed dual-process cognitive architecture (Clarion), see:
Sun, R. (2016). Anatomy of the Mind<https://www.google.com/url?q=https%3A%2F%2Fglobal.oup.com%2Facademic%2Fproduct%2Fanatomy-of-the-mind-9780199794553%3Fcc%3Dus%26lang%3Den%26&sa=D&sntz=1&usg=AFQjCNG6Ilz6RHmCncGzw4gkvbscDKcD5w>. Oxford University Press, New York.
And also some downloadable journal papers about the cognitive architecture:
*
S. Helie and R. Sun, Incubation, insight, and creative problem solving: A unified theory and a connectionist model. Psychological Review, Vol.117, No.3, pp.994-1024. 2010. [formatted PDF<https://www.google.com/url?q=https%3A%2F%2Fwww.dropbox.com%2Fs%2F4mqmh0iboa231yb%2Fhelie-sun-psycrev2010-f.pdf%3Fdl%3D0&sa=D&sntz=1&usg=AFQjCNGtR80X-RnSLqNvRxTllhvHnyGkNA>] [Web<http://www.google.com/url?q=http%3A%2F%2Fccn.psych.purdue.edu%2Fpapers%2Fpr-eii%2Findex.htm&sa=D&sntz=1&usg=AFQjCNGVL8q_vhzC-vnIhyBN7w87IJ6-RQ>]
*
R. Sun, P. Slusarz, and C. Terry, The interaction of the explicit and the implicit in skill learning: A dual-process approach . Psychological Review, Vol.112, No.1, pp.159-192. 2005. [formatted PDF<https://www.google.com/url?q=https%3A%2F%2Fwww.dropbox.com%2Fs%2Fty9aualxu3dbm1m%2Fsun-pr2005-f.pdf%3Fdl%3D0&sa=D&sntz=1&usg=AFQjCNH4tg6vZKL3edLT1ZNTBcQOnyKhiQ>]
*
S. Bretz and R. Sun, Two models of moral judgment. Cognitive Science, Vol.42, No.S1, pp.4-37. 2018. https://onlinelibrary.wiley.com/doi/pdf/10.1111/cogs.12517<https://www.google.com/url?q=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2Fpdf%2F10.1111%2Fcogs.12517&sa=D&sntz=1&usg=AFQjCNECUVGtkcM_ufSdIuVQZDN4Jl73tA>
etc.
Furthermore, for a detailed comparison of this dual-process theory (as embodied in the cognitive architecture) and Kahnenman’s that highlighted their differences, see the analysis in:
R. Sun, Implicit and explicit processes: Their relation, interaction, and competition. In: L. Macchi, M. Bagassi, and R. Viale, (eds.), Cognitive Unconscious and Human Rationality. pp.257-274. MIT Press, Cambridge, MA. 2016.
Best regards,
Ron
========================================================
Professor Ron Sun, Ph.D., FIEEE, FAPS, FPsyS
Cognitive Science Department
Rensselaer Polytechnic Institute
110 Eighth Street, Carnegie 302A
Troy, NY 12180, USA
phone: 518-276-3409
fax: 518-276-3017
email: dr.ron.sun [AT] gmail.com<http://gmail.com/>
web: http://sites.google.com/site/drronsun
=======================================================
On Dec 23, 2022, at 12:00 PM, connectionists-request at mailman.srv.cs.cmu.edu<mailto:connectionists-request at mailman.srv.cs.cmu.edu> wrote:
Date: Thu, 22 Dec 2022 20:00:44 +0000
From: "Park, Deok Gun" <deokgun.park at uta.edu<mailto:deokgun.park at uta.edu>>
To: Stephen Jos? Hanson <jose at rubic.rutgers.edu<mailto:jose at rubic.rutgers.edu>>,
"connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>"
<connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu>>
Subject: Re: Connectionists: Yann Le Cun and Stephen Hanson on Yann's
new vision of AI
Message-ID:
<BY3PR01MB66604501EC4E6E2E2386B2FB9FE69 at BY3PR01MB6660.prod.exchangelabs.com<mailto:BY3PR01MB66604501EC4E6E2E2386B2FB9FE69 at BY3PR01MB6660.prod.exchangelabs.com>>
Content-Type: text/plain; charset="windows-1252"
Thank you, Hanson, for sharing this.
It is inspiring to see a non-homogeneous cognitive architecture from mainstream researcher. If anyone is pursuing similar approaches, please let me know so that I can learn from it.
I was pursuing similar direction myself. My version also has a hierarchical (heterarchical) prediction memory and simulation device for mode 2 thinking.
Below is the link to the position paper. Like Yann?s, this is more of a proposal without results.
https://arxiv.org/abs/2212.08626
If you find this paper interesting, I would be happy to have a seminar or meeting for discussion. I would also appreciate your feedback about my direction.
Thank you for reading.
Regards
===================
Deokgun Park
Assistant Prof. at UTA CSE
===================
From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on behalf of Stephen Jos? Hanson <jose at rubic.rutgers.edu>
Date: Thursday, December 15, 2022 at 8:46 AM
To: connectionists at mailman.srv.cs.cmu.edu <connectionists at mailman.srv.cs.cmu.edu>
Subject: Connectionists: Yann Le Cun and Stephen Hanson on Yann's new vision of AI
[External]
In the last year, Yann Le Cun has posted a new vision for AI, partly avoiding the ambiguous complexity of LLM, and AGI, he focuses on
explicit reasoning systems, going beyond the implicit classification systems which have been so successful with Deep Learning. Like other AI leaders, he realized that somethingthay must be scaled down before scaling up. It would be better to get the reasoning sense of a non-human mammal, say a cat, then jump to human reasoning, Yann argues.
This also underscores, I think, much of the problem with LLM--e.g. ChatBOTs, which probably will be a concern for classicial lingustics, nonetheless are inscructable.
At the same time critics have emerged, because when one provides enough detail of a specific vision it is both courageous and risky. Our favorite critic/troll, Gary Marcus wrote several critiques (see the one below), and Yann and Jake Browning tried to clarify, and deflate some of the Marcus arguments, as many of them appear to almost perversely misunderstand the difference between complex Nerual Network architectures and symbolic systems from the 1970s (which mostly didn't work).
Here is the conversation in whole focused entirely on Yann's new AI vision. Cheers and Happy Holidays--Steve
https://www.youtube.com/watch?v=4s2S2l2OvdM&t=32s<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D4s2S2l2OvdM%26t%3D32s&data=05%7C01%7Cdeokgun.park%40uta.edu%7C47a74d0fd564406c63b108dadeab2720%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C0%7C0%7C638067123945237532%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4f%2FPr%2FoFN6j5Cn7rvREd%2Fp5QonMfOGrBxUUZMNXKr%2B4%3D&reserved=0>
The description of the paper and critques.
https://ai.facebook.com/blog/yann-lecun-advances-in-ai-research/<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fai.facebook.com%2Fblog%2Fyann-lecun-advances-in-ai-research%2F&data=05%7C01%7Cdeokgun.park%40uta.edu%7C47a74d0fd564406c63b108dadeab2720%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C0%7C0%7C638067123945393752%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=My0Iz%2BCbElB1aSjdnUxhpnYmq0swoV7YM%2BhjsHkWJ8U%3D&reserved=0>
https://www.technologyreview.com/2022/06/24/1054817/yann-lecun-bold-new-vision-future-ai-deep-learning-meta/<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.technologyreview.com%2F2022%2F06%2F24%2F1054817%2Fyann-lecun-bold-new-vision-future-ai-deep-learning-meta%2F&data=05%7C01%7Cdeokgun.park%40uta.edu%7C47a74d0fd564406c63b108dadeab2720%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C0%7C0%7C638067123945393752%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DkNduRAz9lBWjPNCHC1qFsKWf6Z7WU6uVJj5qHRbIRQ%3D&reserved=0>
https://garymarcus.substack.com/p/how-new-are-yann-lecuns-new-ideas<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fhow-new-are-yann-lecuns-new-ideas&data=05%7C01%7Cdeokgun.park%40uta.edu%7C47a74d0fd564406c63b108dadeab2720%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C0%7C0%7C638067123945393752%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1zHD1mjZYAI8Wy3V2rHIDPpKAdnu00CvgUuOd1KFQuI%3D&reserved=0>
https://www.noemamag.com/what-ai-can-tell-us-about-intelligence/<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.noemamag.com%2Fwhat-ai-can-tell-us-about-intelligence%2F&data=05%7C01%7Cdeokgun.park%40uta.edu%7C47a74d0fd564406c63b108dadeab2720%7C5cdc5b43d7be4caa8173729e3b0a62d9%7C0%7C0%7C638067123945393752%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gi4YYlKf2%2B5SxbiJnou8sdMG6hst%2FelbhP4fUMKGYE%3D&reserved=0>
*(due to a small mic issue, we have also closed captioned the conversation)
--
Stephen Jos? Hanson
Professor, Psychology Department
Director, RUBIC (Rutgers University Brain Imaging Center)
Member, Executive Committee, RUCCS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20221222/9e1edc56/attachment-0001.html>
------------------------------
Message: 3
Date: Thu, 22 Dec 2022 21:31:28 +0000
From: Stephen Jos? Hanson <jose at rubic.rutgers.edu>
To: "Park, Deok Gun" <deokgun.park at uta.edu>,
"connectionists at mailman.srv.cs.cmu.edu"
<connectionists at mailman.srv.cs.cmu.edu>
Subject: Re: Connectionists: Yann Le Cun and Stephen Hanson on Yann's
new vision of AI
Message-ID: <d9acae24-72de-6f83-1aad-380ca889d211 at rubic.rutgers.edu>
Content-Type: text/plain; charset="utf-8"
Hi Park
So to be clear, its actually not heterogeneous in a representational sense. That's what is often confabulated by critics.. as soon as a neural architecture.. oh I suppose like a brain.. it must be symbolic! Billions of Neurons can't possibly be reasoning.
So to the extent Yann's system can monitor an error signal , a mismatch, a credit reconciliation.. backpropagation will still work fine over a set of homogeneous representaional elements, like neurons.
Steve (my last name is Hanson, I suspect I just made the same error with you!)
On 12/22/22 15:00, Park, Deok Gun wrote:
Thank you, Hanson, for sharing this.
It is inspiring to see a non-homogeneous cognitive architecture from mainstream researcher. If anyone is pursuing similar approaches, please let me know so that I can learn from it.
I was pursuing similar direction myself. My version also has a hierarchical (heterarchical) prediction memory and simulation device for mode 2 thinking.
Below is the link to the position paper. Like Yann?s, this is more of a proposal without results.
https://arxiv.org/abs/2212.08626<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F2212.08626&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264194506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=sAUihmwXTzRIJDUp7LTSloLsRw%2BchaQvt1oiOn0%2Fxgc%3D&reserved=0>
If you find this paper interesting, I would be happy to have a seminar or meeting for discussion. I would also appreciate your feedback about my direction.
Thank you for reading.
Regards
===================
Deokgun Park
Assistant Prof. at UTA CSE
===================
From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu><mailto:connectionists-bounces at mailman.srv.cs.cmu.edu> on behalf of Stephen Jos? Hanson <jose at rubic.rutgers.edu><mailto:jose at rubic.rutgers.edu>
Date: Thursday, December 15, 2022 at 8:46 AM
To: connectionists at mailman.srv.cs.cmu.edu<mailto:connectionists at mailman.srv.cs.cmu.edu> <connectionists at mailman.srv.cs.cmu.edu><mailto:connectionists at mailman.srv.cs.cmu.edu>
Subject: Connectionists: Yann Le Cun and Stephen Hanson on Yann's new vision of AI
[External]
In the last year, Yann Le Cun has posted a new vision for AI, partly avoiding the ambiguous complexity of LLM, and AGI, he focuses on
explicit reasoning systems, going beyond the implicit classification systems which have been so successful with Deep Learning. Like other AI leaders, he realized that somethingthay must be scaled down before scaling up. It would be better to get the reasoning sense of a non-human mammal, say a cat, then jump to human reasoning, Yann argues.
This also underscores, I think, much of the problem with LLM--e.g. ChatBOTs, which probably will be a concern for classicial lingustics, nonetheless are inscructable.
At the same time critics have emerged, because when one provides enough detail of a specific vision it is both courageous and risky. Our favorite critic/troll, Gary Marcus wrote several critiques (see the one below), and Yann and Jake Browning tried to clarify, and deflate some of the Marcus arguments, as many of them appear to almost perversely misunderstand the difference between complex Nerual Network architectures and symbolic systems from the 1970s (which mostly didn't work).
Here is the conversation in whole focused entirely on Yann's new AI vision. Cheers and Happy Holidays--Steve
https://www.youtube.com/watch?v=4s2S2l2OvdM&t=32s<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D4s2S2l2OvdM%26t%3D32s&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264194506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=woH84FLS7JXNThQYw9uhqLZdH8iLyCYkfQAboD7f28g%3D&reserved=0>
The description of the paper and critques.
https://ai.facebook.com/blog/yann-lecun-advances-in-ai-research/<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fai.facebook.com%2Fblog%2Fyann-lecun-advances-in-ai-research%2F&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264350753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=etkNIQNxE9WkuIVDnzkFyTbc8EkjJZN7Z1%2FSkxwlHwo%3D&reserved=0>
https://www.technologyreview.com/2022/06/24/1054817/yann-lecun-bold-new-vision-future-ai-deep-learning-meta/<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.technologyreview.com%2F2022%2F06%2F24%2F1054817%2Fyann-lecun-bold-new-vision-future-ai-deep-learning-meta%2F&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264350753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=bdEu9EVczOkilXR0fubFeiUuGobJD%2BSCzr37g4lzEFA%3D&reserved=0>
https://garymarcus.substack.com/p/how-new-are-yann-lecuns-new-ideas<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fhow-new-are-yann-lecuns-new-ideas&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264350753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=wwKbuJyFKOn27u7aongD9LHulbfsVgzA6KM8CSfQx8g%3D&reserved=0>
https://www.noemamag.com/what-ai-can-tell-us-about-intelligence/<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.noemamag.com%2Fwhat-ai-can-tell-us-about-intelligence%2F&data=05%7C01%7Cjose%40rubic.rutgers.edu%7C866a5fe7a0974c1eb0f408dae45737b4%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C638073361264350753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=XZK7vQJDJRiuVxSPnDkYSD1oB9OUxacQzkWxpwFLaYw%3D&reserved=0>
*(due to a small mic issue, we have also closed captioned the conversation)
--
Stephen Jos? Hanson
Professor, Psychology Department
Director, RUBIC (Rutgers University Brain Imaging Center)
Member, Executive Committee, RUCCS
--
Stephen Jos? Hanson
Professor, Psychology Department
Director, RUBIC (Rutgers University Brain Imaging Center)
Member, Executive Committee, RUCCS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20221222/d5ece55e/attachment-0001.html>
------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20221226/a1b22ea7/attachment.html>
More information about the Connectionists
mailing list