[ACT-R-users] Ref for empirical tests of cognitive models by predicting task difficulty

[Frank Ritter]

the Diag model with Bibby and the followup work by Friedrich both 
predict with models not only which problems will be more difficult, 
but also the transfer across problems.   The work with St. Amant 
shows interface differences

http://acs.ist.psu.edu/papers/ritterBip.pdf
Ritter, F. E., & Bibby, P. A. (2008). Modeling how, when, and what 
learning happens in a diagrammatic reasoning task. Cognitive Science. 
 32, 862-892. [This paper does not cite a paper by Bovair et al. 
(1990), and it should. Not because ideas were taken from it, at least 
explicitly, but because the Bovair work is similar and would be 
useful to anyone going further in this area {The acquisition and 
performance of text-editing skill: A cognitive complexity analysis. 
Human-Computer Interaction, 5, 1-48.}]

http://acs.ist.psu.edu/papers/friedrichR09.pdf
Friedrich, M. B., & Frank E. Ritter, F. E. (2009).  Reimplementing a 
diagrammatic reasoning model in Herbal.  In Proceedings of ICCM - 
2009- Ninth International Conference on Cognitive Modeling. 438-439. 
Manchester, England.

http://acs.ist.psu.edu/papers/stamantHR07.pdf
St. Amant, R., Horton, T. E., & Ritter, F. E. (2007). Model-based 
evaluation of expert cell phone menu interaction. ACM Transactions on 
Computer-Human Interaction, 14(1), Article 1 (May 2007), 24 pages.

cheers,

Frank

At 12:04 -0400 28/4/14, Bonnie E John wrote:
>Oh, I forgot to mention, Leonghwee's work is with CogTool-Explorer, 
>which predicts the errors of novice users, so it addresses Ken's 
>desire for error rates.
>Of course, all the "normal" CogTool work predicts differences in 
>task execution time across tasks and UIs, as does all the 
>Keystroke-Level Model and GOMS work before and since CogTool. So if 
>you want details of KLM predictions, I'd start a The Psychology of 
>Human-Computer Interaction by Card, Moran and Newell and work 
>forwards. That book has dozens of examples with match to data.
>
>  Ken Koedinger ---04/28/2014 09:51:49 AM---I'm looking for 
>references for the following statement and figure the  ACT-R 
>community may have some
>
>From:	Ken Koedinger <koedinger at cmu.edu>
>To:	act-r-users at actr-server.hpc1.cs.cmu.edu
>Date:	04/28/2014 09:51 AM
>Subject:	[ACT-R-users] Ref for empirical tests of cognitive 
>models by	predicting task difficulty
>Sent by:	"ACT-R-users" <act-r-users-bounces at actr-server.hpc1.cs.cmu.edu>
>
>
>
>
>
>I'm looking for references for the following statement and figure the
>ACT-R community may have some:
>
>"One way to empirically evaluate the quality of a cognitive model is to
>test whether it can be used to accurately predict differences in task
>difficulty."
>
>I'm particularly interested in references to models that predict error
>rates (but reaction time prediction is ok too) across a number of
>related tasks.  Models that predict errors at steps in tasks and/or
>specific strategy or error differences are even better.  One such
>reference is our own tech report below -- see constraint C3 in Table 1.
>
>Koedinger, K.R., & MacLaren, B. A. (2002).Developing a pedagogical
>domain theory of early algebra problem solving.CMU-HCII Tech Report
>02-100.[PDF
><<http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf>http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf>]
>
>Others?  Including your own work?
>
>Thanks!
>Ken
>
>
>_______________________________________________
>ACT-R-users mailing list
>ACT-R-users at act-r.psy.cmu.edu
><https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users>https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users
>
>
>_______________________________________________
>ACT-R-users mailing list
>ACT-R-users at act-r.psy.cmu.edu
>https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users