Connectionists: Call for Papers -- NIPS '13 Workshop on Crowdsourcing: Theory, Algorithms, and Applications

[Qiang Liu]

NIPS '13 Workshop on Crowdsourcing: Theory, Algorithms, and Applications

December 9, 2013 (Tentative)
Lake Tahoe, Nevada, USA

http://www.ics.uci.edu/~qliu1/nips13_workshop

Submission Deadline: October 15, 2013
Author Notification: October 30, 2013

***************************************************************************

Machine learning systems involve an integration of data representing human
or physical knowledge, and algorithms that discover patterns in this data
and make predictions about new instances. While machine learning research
usually focuses on developing more efficient learning algorithms, it is
often the quality and amount of training data that predominately govern the
performance of real-world systems. This is only amplified by the recent
popularity of large scale and complicated learning systems such as deep
networks, which can require millions to billions of training instances to
perform well. Unfortunately, traditional methods of collecting data from
specialized workers are usually expensive and slow. In recent years,
however, a potential for change has emerged thanks to crowdsourcing, which
enables huge amounts of labeled data to be collected from large groups of
(usually online) workers for a low cost or no cost at all. Many machine
learning tasks, such as computer vision and natural language processing,
increasingly benefit from data gathered on crowdsourcing platforms such as
Amazon Mechanical Turk and CrowdFlower. On the other hand, tools in machine
learning, game theory, and mechanism design can help to address many
challenging problems in crowdsourcing systems, such as making them more
reliable, more efficient, and less expensive.

In this workshop, we call attention to crowdsourcing as a source of data,
discussing cheap and fast data collection methods based on crowdsourcing,
and how these methods could impact subsequent stages of machine learning.
Furthermore, we will emphasize how the data sourcing paradigm interacts
with the most recent emerging trends in the NIPS community.

Examples of topics of interest in the workshop include (but are not limited
to):

* Applications of crowdsourcing to machine learning

* Reliable crowdsourcing, e.g., label aggregation, quality control

* Optimal budget allocation or active learning in crowdsourcing

* Pricing and incentives in crowdsourcing markets

* Workflow design and answer aggregation for complex tasks (e.g., machine
translation, proofreading)

* Prediction markets / information markets and their connection to learning

* Theoretical analyses of crowdsourcing algorithms, e.g., error rates and
sample complexities for label aggregation and budget allocation algorithms


Invited Speakers
~~~~~~~~~~~~~~~~~~
TBD


Submission Details
~~~~~~~~~~~~~~~~~

Submissions should follow the NIPS format (
http://nips.cc/Conferences/2013/PaperInformation/AuthorSubmissionInstructions)
and are encouraged to be up to eight pages, excluding references. Papers
submitted for review do not need to be anonymized. There will be no
official proceedings, but the accepted papers will be made available on the
workshop website. Accepted papers will be either presented as a talk or
poster. We welcome submissions both on novel research work as well as
extended abstracts on work recently published or under review in another
conference or journal (please state the venue of publication in the later
case); we particularly encourage submission of visionary position papers on
the emerging trends on the field.

Please submit papers in PDF format at
https://cmt.research.microsoft.com/CROWDNIPS2013/.


Workshop Organizers:
~~~~~~~~~~~~~~~~~~~~~

Dengyong Zhou, Jenn Wortman Vaughan, Nikhil R. Devanur. Microsoft Research

Qiang Liu, Alexander Ihler. UC Irvine

Xi Chen. UC Berkeley & NYU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20130822/2b972627/attachment.html>