reviewer comments about cognitive modeling

[Erik M. Altmann]

I'm enjoying the anecdotes, so am forwarding mine to the list.

Among recent attacks in the literature, there is the Cooper and 
Shallice article in Cognition (1995, v. 55, pp. 115-149), "Soar and 
the case for unified theories of cognition".  Their argument is 
roughly that computational theories contain too many assumptions to 
test them all [so we should forgo precision altogether]. Apparently 
Nelson Cowan, in his book, also cites the need to make assumptions as 
a reason to avoid that kind of research (I'd check, but my copy is 
packed...).

Erik.


-- From reviews of "Functional decay in serial attention", submitted 
to JEP: HPP.

Action editor:  "On the one hand, you have generated an empirical 
pattern that occurs regularly, in your studies, so long as bivalent 
stimuli were used (RT increases with increasing trials using the same 
rule, after the first trial using it) and you have generated a 
theoretical framework that appears to accommodate this pattern.  On 
the other hand, your memory decay model was not regarded as plausible 
and your empirical pattern of increasing RTs was not regarded as 
typical for this literature.  With regard to the first general 
criticism, I'm afraid I don't have specific advise for you on how to 
make your model more appealing."

Reviewer B vacillates between dismissing a model he seems not to 
understand very well, and toying with its implications:  "Another big 
problem is that the decay hypothesis has the look and feel of a 
post-hoc attempt to account for the data.  The theoretical 
justification is quite weak and doesn't make sense.  It isn't clear 
why there is an appeal to memory retrieval.  If the productions [sic] 
are already loaded into working memory, then why reload them again? 
And why would it be more difficult to reload them with repetition? 
Also, wouldn't the decay hypothesis suggest that there would be 
minimal switching cost at P1?  What would the decay hypothesis 
predict for a continually increasing pattern on no-switch blocks?"

-- From reviews of "Preparing to forget: Memory and functional decay 
in serial attention", submitted to JEP: General.  This was a slightly 
longer version of the paper above, and submitted first.

Reviewer A comments on an algebraic model that depends on several 
assumptions to make predictions.   He seems to find the model post 
hoc unless it can accommodate the predictions he reads into it:   "I 
think that the 10 encoding cycles and 100 ms cycle time estimate made 
by the authors in Appendix A are very arbitrary and post hoc.  They 
derive from the implicit assumption that subjects adjust the number 
of encoding cycles to the average trials in a sequence (10 in Exp. 1, 
2, and 4). If this assumption is correct, instruction and first trial 
costs should vary with average sequence length.  In Experiment 3, 
average sequence length was 4 trials; were instruction times and 
switch costs reduced accordingly?  If not, the Appendix A assumptions 
are hard to accept."

Reviewer B is more positive:  "I found the model presented in 
Appendix A helpful.  The model depends somewhat on the idea that the 
strength with which a new task set is imposed depends on the 
subjective anticipated run length.  So it would seem to predict that 
for a run of fixed length, more errors should be expected in 
conditions where the subject has reason to expect runs of a shorter 
length.  Although one can imagine other explanations for such a 
pattern, this would seem like a result that would be much easier to 
accommodate within the authors decay-based framework than within a 
more inertia-based account."

-- From reviews of "The anatomy of serial attention:  An integrated 
model of set shifting and maintenance", submitted to Psych. Science.

The action editor, speaking for the reviewers, makes a fair point: 
"While all [three reviewers] agree that there is much of merit in the 
empirical work and in the modeling approach taken by the authors, 
they are also in strong agreement on the ways in which the present 
paper is not ready for publication.  I won't summarize all of these 
points here, but some of the most important are (a) the opinion that 
the behavioral task described represents a somewhat restricted range 
of task switching, (b) a general dissatisfaction with the level of 
detail that is possible for data and models of this kind in a short 
report, (c) serious doubt that the most important of the new findings 
are observed generally in other task switching situations, and (d) a 
lack of attention to or misreading of some relevant antecedent 
literature."

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Erik M. Altmann
Department of Psychology
Michigan State University
East Lansing, MI  48824
517-353-4406 (voice)
703-993-1326 (voice, through July 15, 2000)
517-353-1652 (fax)
ema at msu.edu
http://www.msu.edu/~ema
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~