[AI Seminar] AI Seminar sponsored by Apple -- Vaishnavh Nagarajan -- Nov 21
Adams Wei Yu
weiyu at cs.cmu.edu
Sun Nov 19 08:46:06 EST 2017
Dear faculty and students,
We look forward to seeing you next Tuesday, Nov 21, at noon in NSH 3305 for
AI Seminar sponsored by Apple. To learn more about the seminar series,
please visit the AI Seminar webpage <http://www.cs.cmu.edu/~aiseminar/>.
On Tuesday, Vaishnavh Nagarajan
<http://www.cs.cmu.edu/~vaishnan/home/index.html> will give the following
Title: Gradient Descent GANs are locally stable
Generative modeling, a core problem in unsupervised learning, aims at
understanding data by learning a model that can generate datapoints that
resemble the real-world distribution. Generative Adversarial Networks
(GANs) are an increasingly popular framework that solve this by optimizing
two deep networks, a "discriminator" and a "generator", in tandem.
However, this complex optimization procedure is still poorly understood.
More specifically, it was not known whether equilibrium points of this
system are "locally asymptotically stable" i.e., when initialized
sufficiently close to an equilibrium point, does the optimization procedure
converge to that point? In this work, we analyze the "gradient descent"
form of GAN optimization (i.e., the setting where we simultaneously take
small gradient steps in both generator and discriminator parameters). We
show that even though GAN optimization does not correspond to a
convex-concave game, even for simple parameterizations, under proper
conditions, its equilibrium points are still locally asymptotically stable.
On the other hand, we show that for the recently-proposed Wasserstein GAN
(WGAN), the optimization procedure might cycle around an equilibrium point
without ever converging to it. Finally, motivated by this stability
analysis, we propose an additional regularization term for GAN updates,
which can guarantee local stability for both the WGAN and for the
traditional GAN. Our regularizer also shows practical promise in speeding
up convergence and in addressing a well-known failure mode in GANs called
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the ai-seminar-announce