[ACT-R-users] Advice

[Dan Bothell]

--On Tuesday, February 06, 2007 5:37 PM +0000 Jean-Claude Golovine 
<jcrgolovine at aol.com> wrote:

>
>
> Hello,
>
> I am having a bit of a hard time at the moment. I have been invited to
> undertake a research project that includes ACT-R and write a toolkit to
> perform cognitive operations on UIs. The platform is Java based which
> means I have to communicate with ACT-R (Lisp) through sockets (maybe).
> However, I have an open mind on the matter and my question to you is:
> what is the best way to interface ACT-R from another platform? I have
> read about some work that uses this approach (TCP/IP datagram's) but I
> cannot find anything substantial to get me going. Can you help?
>
>

I would say that there is no universal "best way" because what's
best for any particular situation depends on a lot of factors.
I've seen lots of different mechanisms used which all seemed
"best" for the situation.  Some example mechanisms would be: through
socket connections (either TCP or UDP), by using the Lisp's
foreign function interface to interact directly with the code of
the other system, or through a file-based read/write system,
and I'm sure there are other means people have used as well.

One thing to consider is whether you really need to connect to
the other system or if it's easier to use something like CogTool
(as suggested in another message by Bonnie John), a system like
SegMan (work by Robert St. Amant) or to mock up an interface within
Lisp using either the native Lisp tools or the basic GUI tools
provided by ACT-R (which are somewhat limited but already fully
supported for models).

If you do decide to hook the model up to the external system,
then there are really two pieces to the design.  One is how you
represent things for the model, and the other being how you get
that interaction into and out of ACT-R.

In terms of hooking things up to the model the general mechanism
is to create what's called a "Device" for ACT-R.  This provides
the visual features to the model and receives the motor actions
from the manual system which it can send to the other system.
For the very basic information on the device you can look at the
reference manual available with the newest version of ACT-R 6.
However, that doesn't yet contain all the detailed information
you'll need to actually implement one, so you'll also need to
look at the docs for ACT-R/PM which can be found at:

<http://chil.rice.edu/projects/RPM/docs/index.html>

Then you need to consider how you hook the device up to the
external system, and some of the things to consider would be:

- Does the other system already have an external interface (sockets
  or otherwise)?
- Do you have the ability to modify the other system or is it a
  fixed application?
- What Lisp software are you using, and how comfortable are you with
  its sockets/streams/foreign function interface/multi-threading
  capabilities?
- What kinds of interaction do you need - the basic mouse/keyboard
  operations built into the ACT-R motor module or more advanced things
  like dragging a mouse to select items and double-clicking?

Depending on what you've got and what you need there are lots
of options and what's "best" depends a lot on the situation.

Finally, here are some notes about other things provided with
ACT-R which could be of use in hooking up an external system:

- If you're using ACL w/IDE on Windows the default devices for
ACT-R will generate actual system level mouse/keyboard actions
which can be detected by any application.  So, you may be able
to use those to control the external system and only need to get
the "visual" information into the model.

- The ACT-R environment GUI is connected to the ACT-R system
via a TCP socket.  Thus, there is support built in for using
TCP stream sockets (but not UDP datagrams).


Sorry, I don't have a simple "here's how to do it" answer
for you, but hopefully that provides you with some resources
to consult and things to consider.  If you have any specific
questions about the ACT-R components, feel free to ask me
about them.

Dan