Connectionists: World wide VVTNS series: Wednesday, June 19 at 11am (EDT), Yasaman Bahri , Google DeepMind | Learning and prediction in artificial deep neural networks: scaling, data manifolds, and universality

[David Hansel]

[image: VVTNS.png]
https://www.wwtns.online - on twitter: wwtns at TheoreticalWide

You are cordially invited to the lecture given by

Yasaman Bahri

Google DeepMind

on the topic of

Learning and prediction in artificial deep neural networks:

scaling, data manifolds, and universality

The lecture will be held on zoom on *Wednesday* *June 19, 2024*, at *11:00
am EDT *
To receive the link: https://www.wwtns.online/register-page


*Abstract:  *Developing scientifically-grounded theories for representation
learning and generalization in artificial deep neural networks remains a
grand challenge of fundamental interest to theoretical neuroscience and
machine learning. I will discuss our work on one facet of this challenge —
namely understanding generalization or “scaling laws” in learned neural
networks as a function of basic control variables. I’ll discuss a taxonomy
we develop that classifies different regimes of scaling behavior. We
identify regimes where generalization exhibits universal scaling behavior
and others where it can be traced back to properties of the data and neural
architecture. The theoretical analysis is enabled by leveraging exactly
solvable models of deep neural networks that arise naturally in the limit
of large hidden layers. Along the way, I’ll also discuss our work on these
theoretical models, which have been a useful starting point for theoretical
descriptions of neural network dynamics. Finally, I’ll discuss our findings
connecting generalization in neural networks to properties of the learned
data manifold. I’ll close by discussing future directions and new
hypotheses that emerge from our findings


*About VVTNS : Created as the World Wide Neuroscience Seminar (WWTNS) in
November 2020 and renamed in homage to Carl van Vreeswijk in Memoriam
(April 20, 2022), its aim is to be a platform to exchange ideas among
theoreticians. Speakers have the occasion to talk about theoretical aspects
of their work which cannot be discussed in a setting where the majority of
the audience consists of experimentalists. The seminars, **held on
Wednesdays at 11 am ET,**  are 45-50 min long followed by a discussion. The
talks are recorded with authorization of the speaker and are available to
everybody on our YouTube channel.*
ᐧ
ᐧ
ᐧ
ᐧ


ᐧ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20240616/93b3641f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VVTNS.png
Type: image/png
Size: 41084 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20240616/93b3641f/attachment.png>