[ACT-R-users] sensitivity analyses of ACT-R parameters

Thu Oct 18 14:57:27 EDT 2012

Coty,
	Yes, like Kevin mentioned, we have been working for a few years to
ease the burden of using large-scale resources for our own cognitive model
evaluation. The MindModeling project was born out of this initiative.
MindModeling at Home is a research project that leverages the combined
resources of local computing grids, shared DoD supercomputing clusters and
the computing resources of thousands of volunteers from around the world for
the advancement of cognitive science. The MindModeling at Home project was
originally developed by scientists and engineers within the Air Force
Research Laboratory (711 HPW/RHAC), but has since transitioned to be a
catalyst for collaboration with academia and other partners interested in
the advancement of computational cognitive science research. 

	The MindModeling at Home system provides an infrastructure for easily
submitting a cognitive model for evaluation performance across a range of
contexts utilizing distributed computing practices. Recent advances in the
infrastructure have enabled billions of simulations to be carried out across
thousands of computers producing the equivalent of hundreds of years of
computational work. These simulations have been conducted using a variety of
formalisms including LISP-based models using the ACT-R cognitive
architecture, Quantum Field Theory models authored in Matlab, and even
virtual world learning agents development utilizing the Python language.
Furthermore, to aid in the model evaluation process, the MindModeling
project provides modelers integrated tools for interactively monitoring and
visualizing the performance of models executing within the infrastructure.
Future work is underway to integrate even more formalisms, tools and
languages into the MindModeling system to further expand the system's
cognitive research enabling capabilities.

	We are always looking for new collaborative opportunities with
others in the cognitive modeling community. If you (or others) would be
interested in gaining access to the computational infrastructure please feel
free to contact me.

Jack Harris, PhD
Research Computer Scientist and Cognitive Scientist
Cognitive Models and Agents Branch (711 HPW/RHAC)
Wright-Patterson AFB, Ohio 45433 
(937) 938-3937

-----Original Message-----
From: Gluck, Kevin A Civ USAF AFMC 711 HPW/RHAC 
Sent: Thursday, October 18, 2012 11:30 AM
To: Cleotilde Gonzalez; act-r-users at act-r.psy.cmu.edu; Harris, Jack J Civ
USAF AFMC 711 HPW/RHAC
Cc: 'Jason Harman'; 'Noam Ben-Asher'
Subject: RE: [ACT-R-users] sensitivity analyses of ACT-R parameters

Coty,

We ran sensitivity analyses of the :bll and :ans parameters in David
Reitter's ACT-UP model of the Dynamic Stocks and Flows task.  Those results
and others (including a procedural parameter sweep using Halbruegge's DSF
model) were published in the attached paper.  It may not be precisely what
you are looking for, but it's certainly similar in spirit.  

The broad point and position is that we learn more about our models,
architectures, and systems by performing and reporting explorations of
performance surfaces in the form of systematic sensitivity and necessity
analyses than we do by limiting ourselves to gradient descent searches for
optimal fits or maximum performance levels.  So I'm glad to know you have
some new research underway that also will involve sensitivity analyses.

As you are no doubt aware, these can become computationally intensive.  We
used two DoD HPC centers for the 40 million runs reported in this paper, in
order to get them done in two weeks, rather than the 10 years or so it would
have taken with the machines we were running in our lab.  Since then, Jack
Harris and his MindModeling team have expanded their system so that it
includes both volunteer resources and HPC clusters.  Perhaps Jack will
elaborate.  I'll just say that it's even more flexible and powerful a
capability than it used to be, and it is available to computational
cognitive scientists who find their research programs restricted by
inadequate access to computational resources.

www.mindmodeling.org

Cheers,

Kevin

-----Original Message-----
From: act-r-users-bounces at act-r.psy.cmu.edu
[mailto:act-r-users-bounces at act-r.psy.cmu.edu] On Behalf Of Cleotilde
Gonzalez
Sent: Wednesday, October 17, 2012 3:50 PM
To: act-r-users at act-r.psy.cmu.edu
Cc: 'Jason Harman'; 'Noam Ben-Asher'
Subject: [ACT-R-users] sensitivity analyses of ACT-R parameters

Hello,
  I am working on a project involving sensitivity analyses of ACT-R
parameters d and s.  Would any of you know of any work that has done
something like this?  Would you please refer papers/authors to me? 

thank you,

Coty

_______________________________________________
ACT-R-users mailing list
ACT-R-users at act-r.psy.cmu.edu
http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5624 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/act-r-users/attachments/20121018/3521f513/attachment.bin>