[ACT-R-users] (no subject)

[Hedderik van Rijn]

Adrian,

[Partly overlapping with Dan's answer:]

at last year's workshop, we presented a session on model fitting. You 
might want to have a look at the slides of those presentations, see 
links under "Symposium 2" at:

   http://act-r.psy.cmu.edu/workshops/workshop-2003/schedule.html

I just noticed that my slides are not that readable. A version exported 
to powerpoint format is available here:

   http://viropage.psy.cmu.edu/~rijn/vanrijn03-actrws-parscapes.ppt

> I have a quick newbie question which I wonder if anyone would be kind 
> enough to help
> me out with. When fitting models to data it seems quite common to 
> estimate one or two
> parameters in the model to maximise the fit; the fewer the better. 
> What is the best or
> the conventional method for doing this? The online tutorials imply 
> that the
> recommended approach is to try out different values until you get a 
> good fit. But I'm
> concerned that with several parameters varying simultaneously I may 
> not happen
> across the optimal fit.

Given sufficient time, one can probably find the best fitting 
parameters by brute force, but most of the time, plugging in parameters 
used in other/similar models gives a reasonably good fit to start out 
with. For presentation purposes, scanning the regions around that 
particular fit will probably yield a fit that is good enough to report.

Another issue is of course what defines an optimal fit. Obviously, a 
"best fit" to a particular data set might not be the best fit for a 
rerun of the experiment that provided the original data. Therefore, one 
should probably aim for a set of parameter values that fits multiple, 
similar data sets. An example of that is to first optimize a fit on an 
existing data set - and than applying those parameters to a new data 
set.

And about the theoretical motivations, it seems that not all parameters 
are equal. It's generally considered OK to search for good fitting F 
and f values in the latency equation, but most of the people in the 
ACT-R community seem to stick to a decay (d) of .5 (but see the work 
of, for example, Pavlik about different ways of approaching decay). In 
similar fashion, modifying W as become a default method of modeling 
certain individual differences - so searching for a good fitting W for 
those differences is in a way motivated. It seems to be one of those 
things that is part of the community's knowledge, but hard to describe 
precisely. Going through ACT-R workshop presentations and the papers 
reported at http://act-r.psy.cmu.edu/publications/ might be a good way 
to get an idea for what is commonly done.

  - Hedderik.