[ACT-R-users] Ref for empirical tests of cognitive models by predicting task difficulty

[Susan Chipman]

         A name has come to mind that is probably the name of the ETS
researcher I mentioned to Ken re efforts to predict the difficulty of math
test items -- Isaac Bejar (pronounced Behar).  Probably there were other
such efforts at ETS for obvious reasons.  However, that work did not
involve cognitive modeling.
          Frank Ritter's response reminded me that David Kieras (quite
possibly in collaboration with Bovair) did some very impressive work on
transfer of training operational procedures for a mythical machine.  Very
precise prediction of learning times in relation to order of training and
overlapping production rules was achieved.  I used graphs showing these
results in briefings at ONR to impress the physical scientists and
engineers that we really were doing SCIENCE.

Susan Chipman


On Tue, Apr 29, 2014 at 6:30 AM, GUNZELMANN, GLENN F DR-03 USAF AFMC 711
HPW/RHAC <glenn.gunzelmann at us.af.mil> wrote:

>
> Here's a paper that looks at different types of problems in the Tower of
> Hanoi that vary in difficulty, with a model that predicts exploratory/final
> path behavior:
>
> http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2003-gunzelmann_an
> derson.pdf
>
> Here's a second, with a model with three different strategies, one of which
> accounts for performance across seven different instrument-flight maneuvers
> of varying difficulty in a UAV simulation:
>
> http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2010-myers_gluck_g
> unzelmann_krusmark.pdf
>
> Finally, in case it is relevant when task difficulty varies as a function
> of
> the individual's state (i.e., well-rested versus sleep-deprived), this one
> looks at errors (and RT) in a simple addition/subtraction task:
>
> http://palm.mindmodeling.org/~glenng/pubs/journalarticles/2012-gunzelmann_mo
> ore_gluck_dinges.pdf
>
> -Glenn
>
>
> -----Original Message-----
> From: ACT-R-users [mailto:act-r-users-bounces at ACTR-SERVER.HPC1.CS.cmu.edu]
> On Behalf Of Ken Koedinger
> Sent: Monday, April 28, 2014 9:50 AM
> To: act-r-users at ACTR-SERVER.HPC1.CS.cmu.edu
> Subject: [ACT-R-users] Ref for empirical tests of cognitive models by
> predicting task difficulty
>
>
> I'm looking for references for the following statement and figure the ACT-R
> community may have some:
>
> "One way to empirically evaluate the quality of a cognitive model is to
> test
> whether it can be used to accurately predict differences in task
> difficulty."
>
> I'm particularly interested in references to models that predict error
> rates
> (but reaction time prediction is ok too) across a number of related tasks.
> Models that predict errors at steps in tasks and/or specific strategy or
> error differences are even better.  One such reference is our own tech
> report below -- see constraint C3 in Table 1.
>
> Koedinger, K.R., & MacLaren, B. A. (2002).Developing a pedagogical domain
> theory of early algebra problem solving.CMU-HCII Tech Report 02-100.[PDF
> <http://pact.cs.cmu.edu/koedinger/pubs/Koedinger,%20McLaren%20.pdf>]
>
> Others?  Including your own work?
>
> Thanks!
> Ken
>
>
> _______________________________________________
> ACT-R-users mailing list
> ACT-R-users at act-r.psy.cmu.edu
> https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users
>
> _______________________________________________
> ACT-R-users mailing list
> ACT-R-users at act-r.psy.cmu.edu
> https://mailman.srv.cs.cmu.edu/mailman/listinfo/act-r-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20140429/e41507ea/attachment.html>