Fwd: Thesis Defense - Steven Dang

[Artur Dubrawski]

This may be of interest to some of us.

Artur

---------- Forwarded message ---------
From: Steven Dang <stevenda at andrew.cmu.edu>
Date: Thu, Dec 2, 2021 at 5:11 PM
Subject: Thesis Defense - Steven Dang
To: hcii-members at cs.cmu.edu <hcii-members at cs.cmu.edu>, Kenneth
Koedinger <kk1u at andrew.cmu.edu>, Jeffrey Bigham <jbigham at cs.cmu.edu>,
Queenie Kravitz <sk5u at andrew.cmu.edu>, John Stamper
<jstamper at cs.cmu.edu>, Geoff Kaufman <gfk at cs.cmu.edu>, Artur Dubrawski
<awd at cs.cmu.edu>, Sidney DMello <Sidney.Dmello at colorado.edu>


Exploring Behavioral Measurement Models of Learner Motivation

**HYBRID EVENT**

Steven Dang
HCII Ph.D. Thesis Defense

Time and Location:

GHC 6115 and Streaming via Zoom

Wednesday, Dec 15th, 10:30am-12:30am (EST)
Meeting ID: 92367656582
Passcode: 8mHPw165

Thesis Committee:
Ken Koedinger (HCII, CMU)
John Stamper (HCII, CMU)
Geoff Kaufman (HCII, CMU)
Artur Dubrawski (RI, CMU)

Sidney D’Mello (CS, University of Colorado, Boulder)

Abstract:

No learning happens until students make the choice to engage,
regardless of how well-designed and personalized a lesson. While
advanced algorithms have been developed to personalize and accelerate
learning, similar highly quality models and algorithms aren’t
available for intelligent support of student motivation. The greatest
challenge lies in the lack of high-quality measurement models to
support the administration of motivational interventions. Existing
models focus on the observable engagement behaviors of students. These
measures are prone to noise from non-learner-specific influences as
opposed to reflecting the underlying motivational drivers of
engagement. This complicates the task of leveraging these analytics
for assessing individual student motivational needs to support greater
engagement.



In this dissertation, I address this gap through the development of a
model to measure student diligence, their capacity to self-regulate
and engage with learning activities. I leverage prior research in
psychology and psychometrics to identify behavioral metric candidates.
Through secondary analysis of a year-long longitudinal dataset of log
data from students learning with intelligent tutoring systems, I
evaluate the viability of these behavioral measures to estimate
student diligence. I further develop these measures by leveraging
theory to account for some cognitive, temporal, and social confounding
factors.  My analysis indicates that these behavioral measures, while
better indicators of diligence, are still prone to other sources of
noise that make the measures unreliable.



To address the unique challenges of measurement with observational
data, I explore the viability of diversifying the model inputs by
leveraging multiple operationalizations of diligence for estimation. I
demonstrate that multi-operational models possess more desirable
psychometric properties than any individual measure. Furthermore, I
developed the Learner Engagement Simulator (LEnS) to generate data
that reflects the challenges of estimating motivational constructs due
to unobserved influences from the social and environmental context. My
analysis of the simulated data reinforces the findings with real
student data that multiple operationalizations of diligence increases
the estimator accuracy.



In summary, this thesis makes several contributions. In the learning
sciences, this work contributes to the understanding of how student’s
capacity to self-regulate interacts with the cognitive, temporal, and
social contexts during classroom learning to form patterns of
engagement. To the learning analytics community, this work contributes
a fine-grained, log-data based model of student diligence that can be
leveraged to assess and support students’ engagement. For the
educational data mining community, this work demonstrates the value of
leveraging multiple-operationalizations when building behavior-based
measurement models of motivation. For the human-computer interaction
community, this work develops the LEnS Framework for furthering the
use of simulations to model how motivational constructs drive the
behavior of systems of learners and educational technology within a
learning environment.




-- 

Steven Dang

PhD Student | Human Computer Interaction Institute

Carnegie Mellon University

stevencdang.com