[ACT-R-users] model validation

[Christian Schunn]

There is a paper and excel file here:
http://www.lrdc.pitt.edu/schunn/gof/index.html

that describes your relatively simple choices and the pros/cons of  
each of them.

-Chris

On Sep 14, 2008, at 6:34 AM, Niels Taatgen wrote:

> Hi Fehmida,
> I am not sure whether I fully understand your question, but let me  
> try to answer what I think you are asking.
> First of all, there are no really good metrics to compare model fits  
> to experimental data, I have published papers without mentioning  
> any. Correlation and SD give some indication, and you might list  
> them for completeness, but I often find them quite uninformative (in  
> the sense that the graph that compares model and data gives much  
> more information).
> In general it is not a good idea to apply statistics like Anova to  
> model results. Some people run as many model simulations as they had  
> subjects, and apply the same statistics. It is much better though to  
> just run the model as often as possible. I agree with Rick (whose  
> reply I just saw coming in) that a replication of the experiment is  
> a good sanity check. Or even better: a prediction for a slightly  
> different experiment.
> Niels
> On Sep 14, 2008, at 10:51 AM, Fehmida Hussain wrote:
>
>> Sorry for reposting, I was desperately looking for some advice on  
>> this:
>>
>> hi all,
>>
>> I have a query regarding model validation.
>> I have implemented a few ACT-R 6 models of attentional networks  
>> simulating
>> experimental studies. I have human data available to validate my  
>> model. the
>> human study itself uses statistics like ANOVA to determine  
>> significane of
>> the conditions and interactions between variables.
>>
>> Is is sufficient (from the point of view of model validation ) for  
>> me to
>> just use Correl and SD to validate the model results against the  
>> human data
>> or is it better to repeat the statistical test to show the same
>> significane? What will be considered better from the point of view of
>> defending my thesis?
>>
>> thanks
>> Fehmida
>> _______________________________________________
>> ACT-R-users mailing list
>> ACT-R-users at act-r.psy.cmu.edu
>> http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users
>
> ===============================================
> Niels Taatgen - University of Groningen, Artificial Intelligence
> Also (but not now) at:
> Carnegie Mellon University, Psychology, BH 345B
> web: http://www.ai.rug.nl/~niels     email: taatgen at cmu.edu
> Telephone: +1 412-268-2815 (CMU) +31 50 3636435 (RUG)
> ===============================================
>
>
> _______________________________________________
> ACT-R-users mailing list
> ACT-R-users at act-r.psy.cmu.edu
> http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20080914/4ddd621b/attachment.html>