Connectionists: Call for papers: NIPS-08 Workshop on Model Uncertainty and Risk in Reinforcement Learning

[Pascal Poupart]

Call for papers
NIPS-08 Workshop on Model Uncertainty and Risk in Reinforcement Learning
http://www.cs.uwaterloo.ca/~ppoupart/nips08-workshop.html
Whistler, BC, Canada
December 13, 2008


Important Dates
---------------
* Oct 30: submission deadline
* Nov 4: notification of acceptance


Overview
--------

Reinforcement Learning (RL) problems are typically formulated in terms of 
Stochastic Decision Processes (SDPs), or a specialization thereof, 
Markovian Decision Processes (MDPs), with the goal of identifying an 
optimal control policy. In contrast to planning problems, RL problems are 
characterized by the lack of complete information concerning the 
transition and reward models of the SDP. Hence, algorithms for solving RL 
problems need to estimate properties of the system from finite data. 
Naturally, any such estimated quantity has inherent uncertainty. One of 
the interesting and challenging aspects of RL is that the algorithms have 
partial control over the data sample they observe, allowing them to 
actively control the amount of this uncertainty, and potentially trade it 
off against performance.

Reinforcement Learning as a field of research, has over the past few years 
seen renewed interest in methods that explicitly consider the 
uncertainties inherent to the learning process. Indeed, interest in 
data-driven models that take uncertainties into account, goes beyond RL to 
the fields of Control Theory, Operations Research and Statistics. Within 
the RL community, relevant lines of research may be classified into the 
following (partially overlapping) sub-fields:

1- Bayesian RL. Bayesian methods attempt to explicitly model uncertainties 
using posterior probability distributions, computed using Bayes' rule. 
Such Bayesian modeling may be used in estimating the MDP's transition and 
reward distributions; or in estimating other quantities that are more 
directly related to performance, such as value function and policy 
gradient.

2- Risk sensitive and robust dynamic decision making. These methods use 
information beyond the expected return, to compute policies that are 
robust to inaccuracies in the estimated model. Such quantities include 
quantiles, as well as higher order moments of the return random variable. 
A closely related family of methods use expectations of non-linear 
mappings of the return, as their measures of performance.

3- RL with confidence intervals. This research is concerned with methods 
that employ Frequentist measures of model uncertainties, based on 
confidence intervals. Much of this research is focused on on-line 
algorithms, whose performance is evaluated concurrently with the learning 
process.

4- Applications of risk-aware and uncertainty-aware decision-making. 
Applications in mission critical tasks, finance, and other risk-sensitive 
domains, where uncertainties have to be taken into account, in order to 
establish a level of worst-case performance, or to guarantee a minimum 
level of performance that may be achieved with high probability.

This workshop is aimed at bringing together researchers working in these 
and related fields, allow them to present their current research, and 
discuss possible directions for future work. We intend to focus on 
possible interactions between the sub-fields listed above, as well as on 
interactions with other related fields, which are outside of the current 
RL mainstream.


Workshop format
---------------

This is a one-day workshop consisting of:

1- Invited talks

2- Contributed talks

3- Panel discussions

3.1- Models that work and those that don't: participants will discuss 
specific applications and theoretical models and share experience regarding 
the effectiveness of different approaches. 

3.2- Benchmarks and challenges: discussion of some proposals for sample 
problems that encompass the core challenges of model uncertainty and 
risk sensitive control that could serve as benchmarks and/or challenges.

4- Poster session


Call for Contributions
----------------------

Participants are invited to submit either a technical paper (eight pages 
in the conference format) or an extended abstract (up to two pages) 
describing research relevant to the workshop. Submissions should be sent 
via email to Pascal Poupart at ppoupart at cs.uwaterloo.ca by Oct 30th 
in Postscript, PDF, or MS Word format. Previously published work that is 
reworded, summarized or extended may be submitted to the workshop.  
However, priority will be given to novel work.  If the papers are of 
sufficient quantity and quality, we will seek to publish them as an 
edited book or journal special issue.


Important Dates
---------------

Oct 30: submission deadline
Nov 4: notification of acceptance
Dec 13: workshop in Whistler


Workshop webpage
----------------

http://www.cs.uwaterloo.ca/~ppoupart/nips08-workshop.html


Organizing Committee
--------------------

1- Yaakov Engel (yakiengel at gmail.com)
2- Mohammad Ghavamzadeh (INRIA - Team SequeL, mgh at cs.ualberta.ca),
3- Shie Mannor (McGill University, shie at ece.mcgill.ca)
4- Pascal Poupart (University of Waterloo, ppoupart at cs.uwaterloo.ca)


-- 
------------------------
Pascal Poupart
Assistant Professor
David R. Cheriton School of Computer Science
University of Waterloo
200 University Avenue West
Waterloo, Ontario
Canada N2L 3G1
------------------------
Web: http://www.cs.uwaterloo.ca/~ppoupart
Email: ppoupart at cs.uwaterloo.ca 
Telephone: 1-519-888-4567x36239 
Fax: 1-519-885-1208
------------------------