<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">There is a paper and excel file here:<div><a href="http://www.lrdc.pitt.edu/schunn/gof/index.html">http://www.lrdc.pitt.edu/schunn/gof/index.html</a></div><div><br></div><div>that describes your relatively simple choices and the pros/cons of each of them.</div><div><br></div><div>-Chris</div><div><br><div><div>On Sep 14, 2008, at 6:34 AM, Niels Taatgen wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Fehmida,<div>I am not sure whether I fully understand your question, but let me try to answer what I think you are asking. </div><div>First of all, there are no really good metrics to compare model fits to experimental data, I have published papers without mentioning any. Correlation and SD give some indication, and you might list them for completeness, but I often find them quite uninformative (in the sense that the graph that compares model and data gives much more information). </div><div>In general it is not a good idea to apply statistics like Anova to model results. Some people run as many model simulations as they had subjects, and apply the same statistics. It is much better though to just run the model as often as possible. I agree with Rick (whose reply I just saw coming in) that a replication of the experiment is a good sanity check. Or even better: a prediction for a slightly different experiment.</div><div>Niels<br><div><div>On Sep 14, 2008, at 10:51 AM, Fehmida Hussain wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr"><div>Sorry for reposting, I was desperately looking for some advice on this:</div> <div> </div> <div>hi all,<br><br>I have a query regarding model validation.<br>I have implemented a few ACT-R 6 models of attentional networks simulating<br>experimental studies. I have human data available to validate my model. the<br> human study itself uses statistics like ANOVA to determine significane of<br>the conditions and interactions between variables.<br><br>Is is sufficient (from the point of view of model validation ) for me to<br>just use Correl and SD to validate the model results against the human data<br> or is it better to repeat the statistical test to show the same<br>significane? What will be considered better from the point of view of<br>defending my thesis?<br><br>thanks<br>Fehmida<br></div></div> _______________________________________________<br>ACT-R-users mailing list<br><a href="mailto:ACT-R-users@act-r.psy.cmu.edu">ACT-R-users@act-r.psy.cmu.edu</a><br><a href="http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users">http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users</a><br></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><div style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">===============================================</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">Niels Taatgen - University of Groningen, Artificial Intelligence</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">Also (but not now) at:</span><span class="Apple-converted-space"><span class="Apple-style-span" style="font-size: 12px; "> </span></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">Carnegie Mellon University, Psychology, BH 345B</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">web: <a href="http://www.ai.rug.nl/~niels">http://www.ai.rug.nl/~niels</a> </span><span class="Apple-converted-space"><span class="Apple-style-span" style="font-size: 12px; "> </span></span><span class="Apple-style-span" style="font-size: 12px; ">email: <a href="mailto:taatgen@cmu.edu">taatgen@cmu.edu</a></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">Telephone: +1 412-268-2815 (CMU) +31 50 3636435 (RUG)</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica; font-size: 12px; "><span class="Apple-style-span" style="font-size: 12px; ">===============================================</span></font></div></div><br class="Apple-interchange-newline"></span> </div><br></div></div>_______________________________________________<br>ACT-R-users mailing list<br><a href="mailto:ACT-R-users@act-r.psy.cmu.edu">ACT-R-users@act-r.psy.cmu.edu</a><br>http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users<br></blockquote></div><br></div></body></html>