<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
The Kieras & Bovair work I remember dealt with learning to fire the Starship Enterprise phaser bank. Here are a couple of references.
<div><br>
</div>
<div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><span class="Apple-tab-span" style="white-space:pre"></span>Kieras, D. E., & Bovair, S. (1984). The role of a mental model in learning to operate a device.
<i>Cognitive Science, 8</i>, 255-273. </div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><br>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><span class="Apple-tab-span" style="white-space:pre"></span>Kieras, D. E., & Bovair, S. (1986). The acquisition of procedures from text: A production system analysis of transfer of
training. <i>Journal of Memory and Language, 25</i>, 507-524. </div>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><br>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; ">Those papers may have initiated the thread of "identical elements" production system models of transfer that included the Singley & Anderson book</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span><span style="font-size: 12px; text-indent: -36px; ">Singley, M. K., & Anderson, J. R. (1989).
</span><i style="font-size: 12px; text-indent: -36px; ">Transfer of cognitive skill</i><span style="font-size: 12px; text-indent: -36px; ">. Cambridge, MA: Harvard University Press.</span></div>
<div>
<div style="text-indent: -36px;"><br>
</div>
<div style="text-indent: -36px;">I</div>
<div style="text-indent: -36px;">I</div>
<div>
<div><br>
</div>
<div>I did a paper based specifically on predicting error rates based on such models, and somewhat inspired by the Kieras & Bovair work on mental models</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span><span style="font-size: 12px; text-indent: -36px; ">Pirolli, P. (1991). Effects of examples and their explanations in a lesson on recursion:</span><span style="font-size: 12px; text-indent: -36px; ">
</span><span style="font-size: 12px; text-indent: -36px; ">A production system analysis.
</span><i style="font-size: 12px; text-indent: -36px; ">Cognition and Instruction, 8</i><span style="font-size: 12px; text-indent: -36px; ">, 207-259.</span><span style="font-size: 12px; text-indent: -36px; "> </span></div>
<div><br>
</div>
<div><br>
</div>
<div>I'm not sure how you want to draw the line between measurement and prediction, but here's a paper on combining objective measurement with production system representations to objectively measure difficulty and ability for individual skills in the Lisp
Tutor</div>
<div><br>
</div>
<div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><span class="Apple-tab-span" style="white-space:pre"></span>Pirolli, P., & Wilson, M. (1998). A theory of the measurement of knowledge content, access, and learning.
<i>Psychological Review, 105</i>, 58-82. </div>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><br>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><br>
</div>
<div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; "><br>
</div>
<div><br>
</div>
<div>On Apr 29, 2014, at 9:46 AM, Susan Chipman <<a href="mailto:susan.chipman@gmail.com">susan.chipman@gmail.com</a>></div>
<div> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div dir="ltr">
<div>
<div> A name has come to mind that is probably the name of the ETS researcher I mentioned to Ken re efforts to predict the difficulty of math test items -- Isaac Bejar (pronounced Behar). Probably there were other such efforts at ETS for obvious reasons.
However, that work did not involve cognitive modeling.<br>
</div>
Frank Ritter's response reminded me that David Kieras (quite possibly in collaboration with Bovair) did some very impressive work on transfer of training operational procedures for a mythical machine. Very precise prediction of learning times in
relation to order of training and overlapping production rules was achieved. I used graphs showing these results in briefings at ONR to impress the physical scientists and engineers that we really were doing SCIENCE.<br>
<br>
</div>
Susan Chipman<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Apr 29, 2014 at 6:30 AM, GUNZELMANN, GLENN F DR-03 USAF AFMC 711 HPW/RHAC
<span dir="ltr"><<a href="mailto:glenn.gunzelmann@us.af.mil" target="_blank">glenn.gunzelmann@us.af.mil</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Here's a paper that looks at different types of problems in the Tower of<br>
Hanoi that vary in difficulty, with a model that predicts exploratory/final<br>
path behavior:<br>
<a href="http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2003-gunzelmann_anderson.pdf" target="_blank">http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2003-gunzelmann_an<br>
derson.pdf</a><br>
<br>
Here's a second, with a model with three different strategies, one of which<br>
accounts for performance across seven different instrument-flight maneuvers<br>
of varying difficulty in a UAV simulation:<br>
<a href="http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2010-myers_gluck_gunzelmann_krusmark.pdf" target="_blank">http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2010-myers_gluck_g<br>
unzelmann_krusmark.pdf</a><br>
<br>
Finally, in case it is relevant when task difficulty varies as a function of<br>
the individual's state (i.e., well-rested versus sleep-deprived), this one<br>
looks at errors (and RT) in a simple addition/subtraction task:<br>
<a href="http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2012-gunzelmann_moore_gluck_dinges.pdf" target="_blank">http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2012-gunzelmann_mo<br>
ore_gluck_dinges.pdf</a><br>
<span class="HOEnZb"><font color="#888888"><br>
-Glenn<br>
</font></span>
<div class="im HOEnZb"><br>
<br>
-----Original Message-----<br>
From: ACT-R-users [mailto:<a href="mailto:act-r-users-bounces@ACTR-SERVER.HPC1.CS.cmu.edu">act-r-users-bounces@ACTR-SERVER.HPC1.CS.cmu.edu</a>]<br>
On Behalf Of Ken Koedinger<br>
Sent: Monday, April 28, 2014 9:50 AM<br>
To: <a href="mailto:act-r-users@ACTR-SERVER.HPC1.CS.cmu.edu">act-r-users@ACTR-SERVER.HPC1.CS.cmu.edu</a><br>
Subject: [ACT-R-users] Ref for empirical tests of cognitive models by<br>
predicting task difficulty<br>
<br>
<br>
</div>
<div class="HOEnZb">
<div class="h5">I'm looking for references for the following statement and figure the ACT-R<br>
community may have some:<br>
<br>
"One way to empirically evaluate the quality of a cognitive model is to test<br>
whether it can be used to accurately predict differences in task<br>
difficulty."<br>
<br>
I'm particularly interested in references to models that predict error rates<br>
(but reaction time prediction is ok too) across a number of related tasks.<br>
Models that predict errors at steps in tasks and/or specific strategy or<br>
error differences are even better. One such reference is our own tech<br>
report below -- see constraint C3 in Table 1.<br>
<br>
Koedinger, K.R., & MacLaren, B. A. (2002).Developing a pedagogical domain<br>
theory of early algebra problem solving.CMU-HCII Tech Report 02-100.[PDF<br>
<<a href="http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf" target="_blank">http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf</a>>]<br>
<br>
Others? Including your own work?<br>
<br>
Thanks!<br>
Ken<br>
<br>
<br>
_______________________________________________<br>
ACT-R-users mailing list<br>
<a href="mailto:ACT-R-users@act-r.psy.cmu.edu">ACT-R-users@act-r.psy.cmu.edu</a><br>
<a href="https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users" target="_blank">https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users</a><br>
</div>
</div>
<br>
_______________________________________________<br>
ACT-R-users mailing list<br>
<a href="mailto:ACT-R-users@act-r.psy.cmu.edu">ACT-R-users@act-r.psy.cmu.edu</a><br>
<a href="https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users" target="_blank">https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
ACT-R-users mailing list<br>
<a href="mailto:ACT-R-users@act-r.psy.cmu.edu">ACT-R-users@act-r.psy.cmu.edu</a><br>
https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users<br>
</blockquote>
</div>
<br>
</div>
</body>
</html>