Connectionists: Neurips 2020 workshop on Human in the loop dialogue systems: call for papers

[Y-Lan Boureau]

************************
*Call for papers*
*Workshop Title: **Human in the loop dialogue systems Workshop
@ NeurIPS 2020*


*Website:*        https://sites.google.com/view/hlds-2020/home
*Workshop Date:*  Will fall on between 12/6/2020 – 12/12/2020
*Submission:*
https://cmt3.research.microsoft.com/HLDS2020/Submission/Index
<https://www.google.com/url?q=https%3A%2F%2Fcmt3.research.microsoft.com%2FHLDS2020%2FSubmission%2FIndex&sa=D&sntz=1&usg=AFQjCNG945q1l4ICz1kwV7HrbSUQ5Ydy1A>
*Submission Deadline:*               October 2nd 2020 (Anywhere on Earth)

*Contact:*  *hld-2020-organizers
<hld-2020-organizers at googlegroups.com>@googlegroups.com
<hld-2020-organizers at googlegroups.com>*
*************************
*Description:*
Conversational interaction systems such as Amazon Alexa, Google Assistant,
Apple Siri, and Microsoft Cortana have become very popular over the recent
years. Such systems have allowed users to interact with a wide variety of
content on the web through a conversational interface. Research challenges
such as the Dialogue System Technology Challenges, Dialogue Dodecathlon,
Amazon Alexa Prize and the Vision and Language Navigation task have
continued to inspire research in conversational AI. These challenges have
brought together researchers from different communities such as speech
recognition, spoken language understanding, reinforcement learning,
language generation, and multi-modal question answering.

Unlike other popular NLP tasks, dialogue frequently has humans in the loop,
whether it is for evaluation, active learning or online reward estimation.
Through this workshop we aim to bring together researchers from academia
and industry to discuss the challenges and opportunities in such human in
the loop setups. We hope that this sparks interesting discussions about
conversational agents, interactive systems, and how we can use humans most
effectively when building such systems. We will highlight areas such as
human evaluation setups, reliability in human evaluation, human in the loop
training, interactive learning and user modeling. We also highly encourage
non-English based dialogue systems in these areas.

The one-day workshop will include talks from senior technical leaders and
researchers to share insights associated with training and evaluating
dialogue systems. We also plan on having oral presentations and poster
sessions on the works related to the workshop. Finally, we will end the
workshop with an interactive panel of speakers. As an outcome we expect the
participants from the NeurIPS community to walk away with better
understanding of human in the loop dialogue modeling as well as key areas
of research in this field. Additionally we would like to see discussions
around the unification of human evaluation setups in a standardized way.



*Scope and topics included but not limited to:*

   - Human Evaluation


   - Human evaluation setups and best practices
      - Cost effective setups
      - Reliability and dealing with bias of evaluation results
      - Interpretability of dialogue systems


   - User Modeling


   - Model based evaluation
      - User simulation


   - Human-in-the-loop


   - Online and offline reinforcement learning
      - Active learning
      - Visual and language learning


*Invited Speakers:*

   - Zhou Yu (Columbia University)
   - Alexander Rudnicky (Carnegie Mellon University)
   - Maxine Eskenazi (Carnegie Mellon University)
   - Gokhan Tur (Amazon)
   - Jason Weston (Facebook)
   - Milica Gasic (Heinrich Heine University Düsseldorf)
   - Ankur Parikh (Google)
   - Percy Liang (Stanford)
   - Larry Heck (Samsung)

*Organizers:*

   - Behnam Hedayatnia (Amazon)
   - Rahul Goel (Google)
   - Shereen Oraby (Amazon)
   - Abigail See (Stanford)
   - Chandra Khatri (Uber)
   - Y-Lan Boureau (Facebook)
   - Alborz Geramifard (Facebook)
   - Marylin Walker (UC Santa Cruz)
   - Dilek Hakkani-Tur (Amazon)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20200823/8d2c91a3/attachment.html>