[ACT-R-users] Ref for empirical tests of cognitive models by predicting task difficulty

[Bonnie E John]

Leonghwee Teo's thesis and, more directly, the subsequent CHI paper had
great predictive power for new users of a website. In 36 tasks, it
predicted 93% of the easy tasks (95% of the users would finish successfully
within 3 minutes and 93% of the hard tasks (>75% of humans couldn't
complete within 3 minutes) with 14% false alarms (incorrectly identifying
easy or medium tasks as hard).
Looks like I neglected to put it in the ACT-R repository, so I'll send it
to you in a separate email.
Bonnie





From:	Ken Koedinger <koedinger at cmu.edu>
To:	act-r-users at actr-server.hpc1.cs.cmu.edu
Date:	04/28/2014 09:51 AM
Subject:	[ACT-R-users] Ref for empirical tests of cognitive models by
            predicting task difficulty
Sent by:	"ACT-R-users" <act-r-users-bounces at actr-server.hpc1.cs.cmu.edu>




I'm looking for references for the following statement and figure the
ACT-R community may have some:

"One way to empirically evaluate the quality of a cognitive model is to
test whether it can be used to accurately predict differences in task
difficulty."

I'm particularly interested in references to models that predict error
rates (but reaction time prediction is ok too) across a number of
related tasks.  Models that predict errors at steps in tasks and/or
specific strategy or error differences are even better.  One such
reference is our own tech report below -- see constraint C3 in Table 1.

Koedinger, K.R., & MacLaren, B. A. (2002).Developing a pedagogical
domain theory of early algebra problem solving.CMU-HCII Tech Report
02-100.[PDF
<http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf>]

Others?  Including your own work?

Thanks!
Ken


_______________________________________________
ACT-R-users mailing list
ACT-R-users at act-r.psy.cmu.edu
https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20140428/47f00c77/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20140428/47f00c77/attachment.gif>