cognitive model interfaces paper

[Kevin.Gluck at williams.af.mil]

Conference Program Committee for the Recommended Reading List from the 11th
Conference on Computer-Generated Forces and Behavior Representation.

Great job, Marios and Frank!

-----Original Message-----
From: Frank E. Ritter [mailto:ritter at ist.psu.edu]
Sent: Friday, July 12, 2002 10:30 AM
To: soar-group at umich.edu; act-r-users+ at andrew.cmu.edu;
cal at agent-software.com.au
Cc: ritter at ist.psu.edu
Subject: cognitive model interfaces paper


A paper Mario Avraamides and I wrote on how to improve a particular 
cognitive model's interface was presented at the Computer Generated 
Forces conference, and at the Soar workshop, so many of you have 
already heard about it.  But those, who haven't seen it and should be 
interested in it, are on these mailing lists, so I'm sending out a 
pointer to it as well.

The paper includes an initial task analysis for models' explanatory 
displays.  The results should be interesting to both Soar, Jack, and 
ACT-R interface builders, including those who are just building 
interfaces for their own models.    I append its abstract and URL
-- 

Cheers,

Frank

Using Multidisciplinary Expert Evaluations to Test and Improve Cognitive
Model Interfaces

Marios N. Avraamides
Department of Psychology
The Pennsylvania State University
University Park, PA 16802
+1 (814) 865-4455
marios at psu.edu
Frank E. Ritter
School of Information Sciences and Technology
The Pennsylvania State University
University Park, PA 16802
+1 (814) 865-4453
ritter at ist.psu.edu

Keywords:
Subject-matter experts, situation awareness, cognitive models

Typically, the design of cognitive models has not emphasized the role 
of interfaces for describing the
models' behavior. Models that populate synthetic environments are 
particularly complex and need support in this area.
Using a variety of subject-matter experts we evaluated the use of the 
Situation Awareness Panel (SAP) as a tool for
inspecting the behavior and reasoning of Soar agents in a JSAF 
simulation. We gathered suggestions on how to
improve future implementations of the SAP from experts in a variety 
of disciplines, including military pilots, cognitive
psychologists, an HCI specialist, a logistic specialist, and a 
software designer. Because of their diversity, we found that
all were able to report unique problems with the interface, and thus 
now suggest about twice as many experts be used
for evaluations than were previously suggested and that the experts 
should vary in their perspective. We used their
behavior and reports to develop a task analysis that can be used as a 
general guide for future designs of user interfaces
for cognitive models in general and for the design of interfaces for 
models in synthetic environments in particular. We
suggest this approach of having multiple types of experts review an 
interface as a general method for improving
complex interfaces such as interfaces to cognitive models.

In Proceedings of the 11th Computer-Generated Forces and Behavior 
Representation Conference
pp. 553-562, 02-CGF-100. Orlando, FL: U. of Central Florida 
http://acs.ist.psu.edu/papers/avraamidesR02.pdf