[AI Seminar] AI Seminar sponsored by Apple -- Wen Sun -- Jan. 29th

[Han Zhao]

Dear faculty and students:

We look forward to seeing you next Tuesday, Jan. 29th, at noon in *NSH
3305 *for our first AI Seminar sponsored by Apple. To learn more about the
seminar series, please visit the website.
On Tuesday, Wen Sun will give the following talk:
*Title: Towards Generalization and Efficiency in Reinforcement Learning*

*Abstract*: In classic supervised machine learning, a learning agent
behaves as a passive observer: it receives examples from some external
environment which it has no control over and then makes predictions.
Reinforcement Learning (RL), on the other hand, is fundamentally
interactive : an autonomous agent must learn how to behave in an unknown
and possibly hostile environment, by actively interacting with the
environment to collect useful feedback. One central challenge in RL is how
to explore an unknown environment and collect useful feedback efficiently.
In recent practical RL success stories, we notice that most of them rely on
random exploration which requires large a number of interactions with the
environment before it can learn anything useful.  The theoretical RL
literature has developed more sophisticated algorithms for efficient
learning, however, the sample complexity of these algorithms has to scale
exponentially with respect to key parameters of underlying systems such as
the dimensionality of state vector, which prohibits a direct application of
these theoretically elegant RL algorithms to large-scale applications.
Without any further assumptions, RL is hard, both in practice and in theory.

In this work, we improve generalization and efficiency on RL problems by
introducing  extra sources of help and additional assumptions. The first
contribution of this work comes from improving RL sample efficiency via
Imitation Learning (IL). Imitation Learning reduces policy improvement to
classic supervised learning. We study in both theory and in practice how
one can imitate experts to reduce sample complexity compared to RL
approaches. The second contribution of this work comes from exploiting the
underlying structures of the RL problems via model-based learning
approaches.  While there exist efficient model-based RL approaches
specialized for specific RL problems (e.g., tabular MDPs, Linear Quadratic
Systems), we develop a unified model-based algorithm that generalizes a
large number of RL problems that were often studied independently in the
literature. We also revisit the long standing debate on whether model-based
RL is more efficient than model-free RL from a theoretical perspective, and
demonstrate that model-based RL can be exponentially more sample efficient
than model-free ones, which to the best of our knowledge, is the first that
separates model-based and model-free general approaches.
-----------------------------------------------------------------------------------------------------------------------------
P. S. This talk should be of great interest to many people working on ML/RL
and other related domains, hence I encourage everyone to attend.
-- 

*Han ZhaoMachine Learning Department*


*School of Computer ScienceCarnegie Mellon UniversityMobile: +1-*
*412-652-4404*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/ai-seminar-announce/attachments/20190127/d56ceef3/attachment.html>