[ACT-R-users] experiments to validate behaviour of a cognitive model

[Florian Frische]

Hi all,

I would like to ask you about the amount of experimental data in order to 
validate a model's behaviour.
I think there are several parameters that have an effect on the amount of 
data (suspects, trials) that is needed. 
The following 2 examples should help to describe what this question is 
about:

Example 1: Let's say we have a 2-armed bandit that always returns -1 on 
the one arm and 1 on the other arm. We Would like 
to validate the behaviour of our model towards the behaviour of gamblers 
(The strategy that they use to maximize the overall outcome)

Example 2: We have a complex flight task where a pilot has to avoid a 
thunderstorm. We would like to validate the behaviour 
of our pilot model in this task towards real pilots behaviour.

It is obvious that example 1 is less complex than example 2 and I think 
that we need much more experimental data to get reliable results 
for the second example. I suppose there is a relationship beetween task 
complexity (e.g. independent variables) and the number of suspects/number 
of trials.
But how can I assess how many suspects/trials I need (How many 
participants and how many repeats)?

Thanks a lot for participating in this discussion,

Florian Frische

OFFIS
FuE Bereich Verkehr | R&D Division Transportation
Escherweg 2 - 26121 Oldenburg - Germany
Phone.: +49 441 9722-523
E-Mail: florian.frische at offis.de
URL: http://www.offis.de