[CMU AI Seminar] November 1 at 12pm (NSH 3305 & Zoom) -- Sébastien Bubeck (Microsoft Research) -- Unveiling Transformers with LEGO -- AI Seminar sponsored by SambaNova Systems

[Asher Trockman]

Dear all,

We look forward to seeing you *this Tuesday (11/1)* from *1**2:00-1:00 PM
(U.S. Eastern time)* for the next talk of this semester's *CMU AI Seminar*,
sponsored by SambaNova Systems <https://sambanova.ai/>. The seminar will be
held in NSH 3305 *with pizza provided *and will be streamed on Zoom.

To learn more about the seminar series or to see the future schedule,
please visit the seminar website <http://www.cs.cmu.edu/~aiseminar/>.

On 11/1, *Sébastien Bubeck* (Microsoft Research) will be giving a talk
titled *"**Unveiling Transformers with LEGO**" *to share his recent work on
probing the inner workings of transformers using a synthetic reasoning task.

*Title*: Unveiling Transformers with LEGO

*Talk Abstract*: The discovery of the transformer architecture was a
paradigm shifting event for deep learning. However, these architectures are
arguably even harder to understand than say convolutional neural networks.
In this work we propose a synthetic reasoning task, called LEGO, to probe
the inner workings of transformers. We obtain some insights on multi-head
attention, the effect of pretraining, as well as overfitting issues.
Joint work with Yi Zhang, Arturs Backurs, Ronen Eldan, Suriya Gunasekar,
and Tal Wagner.

*Speaker Bio*: Sebastien Bubeck leads the Machine Learning Foundations
group at Microsoft Research Redmond. He joined MSR in 2014, after three
years as an assistant professor at Princeton University. He received
several best paper awards at machine learning conferences for his work on
online decision making, convex optimization, and adversarial robustness
(NeurIPS 2021, NeurIPS 2018, ALT 2018, COLT 2016, COLT 2009). He also wrote
two monographs, “Regret Analysis of Stochastic and Non-Stochastic
Multi-Armed Bandit Problems” (2012) and “Convex Optimization: Algorithms
and Complexity” (2014).

*In person: *NSH 3305
*Zoom Link*:
https://cmu.zoom.us/j/99510233317?pwd=ZGx4aExNZ1FNaGY4SHI3Qlh0YjNWUT09

Thanks,
Asher Trockman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/ai-seminar-announce/attachments/20221030/3d60386c/attachment.html>