[CL+NLP Lunch] Nov. 15 @ noon: Diane Litman, "Automatically Predicting Peer-Review Helpfulness"

[Benjamin Lambert]

Reminder: this afternoon!

CL+NLP Lunch
Tuesday, Nov. 15 @ noon in GHC 4405
Lunch will be provided.

Diane Litman
Professor, Computer Science
Senior Scientist, Learning Research and Development Center
University of Pittsburgh

Automatically Predicting Peer-Review Helpfulness

One path to improving the quality of student writing has involved the
use of a peer review process, often supported by web-based
technology. The long-term goal of our research is to use Natural
Language Processing to address three core problems in peer-review of
writing: reviews are often stated in ineffective ways, reviews and
papers do not focus on important paper aspects, and authors do not
have a process for organizing paper revisions. This talk will present
our research on automatically predicting the helpfulness of peer
reviews, one important task for improving the quality of feedback
received by students, as well as for helping students write better
reviews.

We first examine whether standard product review analysis techniques
also apply to our new context of peer reviews.  We also investigate
the utility of incorporating additional specialized features tailored
to peer review. Our preliminary results show that structural features,
review unigrams and meta-data are useful in modeling the helpfulness
of both peer reviews and product reviews, while peer-review specific
auxiliary features can further improve helpfulness prediction.
Finally, we investigate how different types of perceived helpfulness
might influence the utility of features for automatic prediction. Our
feature selection results show that certain low-level linguistic
features are more useful for predicting student perceived helpfulness,
while high-level cognitive constructs are more effective in modeling
experts' perceived helpfulness.

This work is done in collaboration with Wenting Xiong, Christian
Schunn, and Kevin Ashley, University of Pittsburgh.

Bio: Diane Litman is Professor of Computer Science, Senior Scientist
with the Learning Research and Development Center, and faculty with
the Graduate Program in Intelligent Systems, all at the University of
Pittsburgh.  She has been working in the field of artificial
intelligence since she received her Ph.D. degree in Computer Science
from the University of Rochester. Before joining Pitt, she was a
member of the Artificial Intelligence Principles Research Department,
AT&T Labs - Research (formerly Bell Laboratories). Dr. Litman's
current research focuses on enhancing the effectiveness of educational
technology through the use of spoken and natural language processing,
affective computing, and machine learning and other statistical
methods.  Dr. Litman has been Chair of the North American Chapter of
the Association for Computational Linguistics, has co-authored
multiple papers winning best paper awards, and has been awarded Senior
Member status by the Association for the Advancement of Artificial
Intelligence.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/nlp-lunch/attachments/20111115/9b5bb825/attachment.html