[ACT-R-users] Model Validation and Model Comparison

[Gluck, Kevin A Civ USAF AFMC 711 HPW/RHAC]

We've seen two recent threads on the topic of model validation.  An
important topic and interesting exchange.

A closely related, but not identical, topic is "model comparison."  Paul
Bello, Jerry Busemeyer, and I recently organized a special issue of the
journal _Cognitive Science_ on the topic of model comparison.  It is in
press and (we think) will appear as the last issue of the journal in
2008.  As a teaser, here is the opening paragraph from our intro to the
special issue:

--------

All cognitive modelers sooner or later (usually sooner) run headlong
into fundamental issues associated with comparing and selecting
alternative models and with evaluating the validity of the models that
have been developed. We developed this special issue around the theme of
Model Comparison in order to highlight the critical importance of these
topics for ongoing progress in cognitive science. What are the different
methods by which one can compare two or more models in terms of their
capacity to explain a set of experimental findings? Can these models be
empirically distinguished in terms of their qualitative and/or
quantitative predictions? What are the prospects for making direct
comparisons across statistical, mathematical, and computational modeling
approaches?  How does the focus of comparisons differ across modeling
approaches and what value might the best practices of one of these
modeling communities have for the others?  How should we deal with the
complex interplay of issues such as the breadth and depth of explanatory
mechanisms, parsimony, plausibility, precision, and accuracy within and
across modeling approaches? Without clear answers to these questions,
how can our science measure its progress? We have assembled a collection
of papers that address some of these and other important questions. 

--------

Hopefully it will be an informative and useful resource.

Cheers,

Kevin

-------------------------------------------------------
KEVIN GLUCK, PhD
Senior Research Psychologist
S&T Advisor, Cognitive Models and Agents Branch
Air Force Research Laboratory
6030 S. Kent St
Mesa, AZ  85212-6061
P: 480-988-6561 x-677; DSN 474-6677
F: 480-988-2230; DSN 474-6688
C: 480-229-4569
 
PALM Webpage:  http://www.mesa.afmc.af.mil/palmlab.html
 
"The true line is not between 'hard' natural science and 'soft' social
sciences, but between precise science limited to highly abstract and
simple phenomena in the laboratory and inexact science and technology
dealing with complex problems in the real world."
                       - Herb Simon, Models of My Life