[ACT-R-users] model validation

[Niels Taatgen]

Hi Fehmida,
I am not sure whether I fully understand your question, but let me try  
to answer what I think you are asking.
First of all, there are no really good metrics to compare model fits  
to experimental data, I have published papers without mentioning any.  
Correlation and SD give some indication, and you might list them for  
completeness, but I often find them quite uninformative (in the sense  
that the graph that compares model and data gives much more  
information).
In general it is not a good idea to apply statistics like Anova to  
model results. Some people run as many model simulations as they had  
subjects, and apply the same statistics. It is much better though to  
just run the model as often as possible. I agree with Rick (whose  
reply I just saw coming in) that a replication of the experiment is a  
good sanity check. Or even better: a prediction for a slightly  
different experiment.
Niels
On Sep 14, 2008, at 10:51 AM, Fehmida Hussain wrote:

> Sorry for reposting, I was desperately looking for some advice on  
> this:
>
> hi all,
>
> I have a query regarding model validation.
> I have implemented a few ACT-R 6 models of attentional networks  
> simulating
> experimental studies. I have human data available to validate my  
> model. the
> human study itself uses statistics like ANOVA to determine  
> significane of
> the conditions and interactions between variables.
>
> Is is sufficient (from the point of view of model validation ) for  
> me to
> just use Correl and SD to validate the model results against the  
> human data
> or is it better to repeat the statistical test to show the same
> significane? What will be considered better from the point of view of
> defending my thesis?
>
> thanks
> Fehmida
> _______________________________________________
> ACT-R-users mailing list
> ACT-R-users at act-r.psy.cmu.edu
> http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users

===============================================
Niels Taatgen - University of Groningen, Artificial Intelligence
Also (but not now) at:
Carnegie Mellon University, Psychology, BH 345B
web: http://www.ai.rug.nl/~niels     email: taatgen at cmu.edu
Telephone: +1 412-268-2815 (CMU) +31 50 3636435 (RUG)
===============================================


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20080914/ac8aa701/attachment.html>