[CMU AI Seminar] Apr 20 at 12pm (Zoom) -- Misha Khodak (CMU) -- Factorized Layers Revisited: Compressing Deep Neural Networks Without Playing the Lottery -- AI Seminar sponsored by Fortive

[Shaojie Bai]

Dear all,

Just a reminder that the CMU AI Seminar <http://www.cs.cmu.edu/~aiseminar/> is
tomorrow *1**2pm-1pm*:
https://cmu.zoom.us/j/93099996457?pwd=b3BSSHp2RWZWQjZ0SUE4ZkdKSDk4UT09.

Misha Khodak (CMU CSD) will be talking about his recent ICLR work on model
compression with layer factorization.

Thanks,
Shaojie

On Thu, Apr 15, 2021 at 12:22 PM Shaojie Bai <shaojieb at andrew.cmu.edu>
wrote:

> Dear all,
>
> We look forward to seeing you *next Tuesday (4/20)* from *1**2:00-1:00 PM
> (U.S. Eastern time)* for the next talk of our *CMU AI seminar*, sponsored
> by Fortive <https://careers.fortive.com/>.
>
> To learn more about the seminar series or see the future schedule, please
> visit the seminar website <http://www.cs.cmu.edu/~aiseminar/>.
> <http://www.cs.cmu.edu/~aiseminar/>
>
> On 4/20, *Misha Khodak* (CMU CSD) will be giving a talk on "*Factorized
> Layers Revisited: Compressing Deep Neural Networks Without Playing the
> Lottery*".
>
> *Title*: Factorized Layers Revisited: Compressing Deep Neural Networks
> Without Playing the Lottery
>
> *Talk Abstract*: Machine learning models are rapidly growing in size,
> leading to increased training and deployment costs. While the most popular
> approach for training compressed models is trying to guess good "lottery
> tickets" or sparse subnetworks, we revisit the low-rank factorization
> approach, in which weights matrices are replaced by products of smaller
> matrices. We extend recent analyses of optimization of deep networks to
> motivate simple initialization and regularization schemes for improving the
> training of these factorized layers. Empirically these methods yield higher
> accuracies than popular pruning and lottery ticket approaches at the same
> compression level. We further demonstrate their usefulness in two settings
> beyond model compression: simplifying knowledge distillation and training
> Transformer-based architectures such as BERT. This is joint work with Neil
> Tenenholtz, Lester Mackey, and Nicolo Fusi.
>
> *Speaker Bio*: Misha Khodaka is a PhD student in Carnegie Mellon
> University's Computer Science Department advised by Nina Balcan and Ameet
> Talwalkar. His research focuses on foundations and applications of machine
> learning, most recently neural architecture search, meta-learning, and
> unsupervised representation learning. He recently spent time as an intern
> with Nicolo Fusi at Microsoft Research - New England and previously
> received an AB in Mathematics and an MSE in Computer Science from Princeton
> University, where he worked with Sanjeev Arora.
>
> *Zoom Link*:
> https://cmu.zoom.us/j/93099996457?pwd=b3BSSHp2RWZWQjZ0SUE4ZkdKSDk4UT09
>
> Thanks,
> Shaojie Bai (MLD)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/ai-seminar-announce/attachments/20210419/a61b5894/attachment.html>