[CMU AI Seminar] April 2 at 12pm (GHC 6115 & Zoom) -- Rafael Frongillo (UC Boulder) -- Incentive problems in data science competitions, and how to fix them -- AI Seminar sponsored by SambaNova Systems

[Asher Trockman]

Dear all,

We look forward to seeing you tomorrow, *this Tuesday (4/2)* from *1**2:00-1:00
PM (U.S. Eastern time)* for the next talk of this semester's
*CMU AI Seminar*, sponsored by SambaNova Systems <https://sambanova.ai/>.
The seminar will be held in GHC 6115 *with pizza provided *and will be
streamed on Zoom.

To learn more about the seminar series or to see the future schedule,
please visit the seminar website <http://www.cs.cmu.edu/~aiseminar/>.

Tomorrow, this Tuesday (4/2), *Rafael Frongillo* (UC Boulder) will be
giving a talk titled *"Incentive problems in data science competitions, and
how to fix them**"*.

*Title*: Incentive problems in data science competitions, and how to fix
them

*Talk Abstract*: Machine learning and data science competitions, wherein
contestants submit predictions about held-out data points, are an
increasingly common way to gather information and identify experts.  One of
the most prominent platforms is Kaggle, which has run competitions with
prizes up to 3 million USD.  The traditional mechanism for selecting the
winner is simple: score each prediction on each held-out data point, and
the contestant with the highest total score wins.  Perhaps surprisingly,
this reasonable and popular mechanism can incentivize contestants to submit
wildly inaccurate predictions.  The talk will begin with a series of
experiments inspired by Aldous (2019) to build intuition for the incentive
issues and what sort of strategic behavior one would expect---and when.
One takeaway is that, despite conventional wisdom, large held-out data sets
do not always alleviate these incentive issues, and small ones do not
necessarily suffer from them, as we confirm with formal results.  We will
then discuss a new mechanism which is approximately truthful, in the sense
that rational contestants will submit predictions which are close to their
best guess.  If time we will see how the same mechanism solves an open
question for online learning from strategic experts.

*Speaker Bio:* Rafael (Raf) Frongillo is an Assistant Professor of Computer
Science at the University of Colorado Boulder.  His research lies at the
interface between theoretical machine learning and economics, primarily
focusing on information elicitation mechanisms, which incentivize humans or
algorithms to predict accurately.  Before Boulder, Raf was a postdoc at the
Center for Research on Computation and Society at Harvard University and at
Microsoft Research New York.  He received his PhD in Computer Science at UC
Berkeley, advised by Christos Papadimitriou and supported by the NDSEG
Fellowship.

*In person: *GHC 6115
*Zoom Link*:  https://cmu
.zoom.us/j/99510233317?pwd=ZGx4aExNZ1FNaGY4SHI3Qlh0YjNWUT09

Thanks,
Asher Trockman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/ai-seminar-announce/attachments/20240401/6c6de3d1/attachment.html>