report on cog modelling conference 98

[ritter at psychology.nottingham.ac.uk]

I was waiting to post this until it came out, and it just came out
this week after a series of misadventures.

I include it mostly because of the Lessons Learned section.

Cheers,

Frank
****************************************************************

  [Young, R. M., & Ritter, F. E. (1999). Report on the Second European
   Conference on Cognitive Modelling. AI and Simulation of Behaviour
   Quarterly, 101, 10-11.]

Report on the Second European Conference on Cognitive Modelling

Richard Young                         Frank E. Ritter
Psychology                            Psychology
U. of Hertfordshire                   University of Nottingham
Hatfield  AL10 9AB                    Nottingham  NG7 2RD
R.M.Young at herts.ac.uk                 Frank.Ritter at nottingham.ac.uk


The Second European Conference on Cognitive Modelling (ECCM-98) was held in
Nightingale Hall at the University of Nottingham from 1 to 4 April 1998.
As well as presented papers, the conference included tutorials (on Act-R,
Soar, and Cogent), invited addresses, symposia, posters, and demonstrations
of models and modelling software.  A complete listing and information on
obtaining the proceedings is available through the conference web pages at
"http://www. psychology. nottingham.ac.uk/staff/ritter/eccm98".

The call for papers encouraged submissions that reported both a running
(i.e. implemented) computer model and some empirical data against which the
model was compared.  We were pleased with the results.  Most of the
submission included both those components, the only real exceptions being
papers where such a criterion was not appropriate, such as those dealing
with tools, architectures, or methodology.
The quantity and the quality were high enough that we were able to be
selective.  Within the constraints of preparing for a conference-where a
large number of papers have to be assessed in a short interval of time, and
where decisions about acceptance or rejection have to be made on the basis
of a paper as it stands-we were able to provide some serious refereeing.
Of course, the review process could not be as thorough as it is for journal
publication, but each paper was read and commented on by at least two
members of the programme committee and a programme co-chair.  We tried hard
to make the feedback given to authors clear and informative, especially in
cases where changes were suggested or where reasons for rejecting a paper
(or accepting it as a poster) were offered.

Of the 40 papers submitted, we accepted 20, and invited a further 10 to be
presented as posters (6 of which took up the invitation).  We also accepted
5 of the 6 poster contributions.  Our main criterion for posters was that
they should be of relevance to the cognitive modelling research community,
but possibly reporting work that was too preliminary to be presented as a
main paper, or possibly focused on a model without as yet including the
comparison to data.  The papers that were published were visibly improved
through the authors' attention to the reviews and their own further
revisions.  The proceedings were published and further information is
available from the web site.

As well as having representation from a wide range of areas of cognitive
modelling, the conference was a truly international event, contributions to
the programme came from 14 different countries: the UK (11), USA (9),
France (8), Germany (7), Italy (3), Belgium (2), Finland (2), The
Netherlands (2), Australia, Bulgaria, Greece, Japan, Sweden, and
Switzerland (1 each).  The proceeding's author index lists no fewer than 80
authors who contributed to the conference.  The conference also had
international sponsorship.  In addition to our departments, support was
provided by the UK Engineering and Physical Science Research Council, the
European Research Office of the US Army and the US Air Force European
Office of Aerospace Research and Development.

Lessons learned

The concentration of models and data was high enough that several new
generalisations emerged.  Models are starting to interact with complex,
interactive task simulations.  This leads to and supports more complex
behaviours, sometimes caused by multiple interacting mechanisms in the
model.

Most models were created within the context of existing architectures, and
that the standard for proposing new architectures is increasing.  The
papers indicated that architectures are being used in new ways, for
example, modifying the architecture to simulate fatigue or cognitive
development.

How many times to run a model that have includes a stochastic component has
been an often asked question in cognitive modelling.  Most papers reported
running the model once per subject modelled.  Having a variety of examples
of this lets one see that the most robust approach is to run the model
until a clear measure of the expected behaviour and range is available.
This gets the most out of the model, for it provides the clearest
comparison and is most likely to indicate where the model can be improved.

Future of the conference

It is appropriate to end this summary with some thoughts about the nature
of the ECCMs and how they relate to other meetings.  Many of us tend to
think of cognitive modelling as a research activity dominated by the USA.
Yet even in the USA, the publication of descriptions of running computer
models and their detailed comparison with empirical data is comparatively
rare, and there seem to be no meetings attempting what ECCM is trying to
do.  The closest that comes to mind is the annual meeting of the Cognitive
Science Society.  Yet the feel of that meeting is entirely different to
ECCM, in part because it is indeed a meeting of a particular scientific
society (which ECCM is not), and in part because Cognitive Science (as
viewed by the Society) is a broad field, of which cognitive modelling is
seen as just a small part.  (Although there are many model+data papers
there, the ratio is not as high and there are many other types of papers as
well.)  What makes ECCM distinctive is the point we stressed above, namely
our emphasis on the presentation of both an implemented model and its
comparison against empirical data, and on keeping a reasonable balance
between the two.

The conference built on the success of the first meeting in the series,
which had been held in Berlin in November 1996.  Its proceedings were
initially published as a technical report of the Berlin University of
Technology, but a subset of the papers have been revised and extended and
are now available as a reasonably priced book (U. Schmid, J. Krems, & F.
Wysotzki (Eds.), Mind modeling -A cognitive science approach to reasoning,
learning and discovery. Lengerich (Germany): Pabst Scientific Publishing,
ISBN 3-933151-25-2, $25/40DM).

There are some uncertainties about future meetings, and especially about
our relationship to the ongoing series of European Conferences on Cognitive
Science (ECCS: St Malo, 1995; Manchester, 1997; Sienna, 1999).  These
matters were discussed at a special session during the conference.  Nothing
was clearly decided about the location and timing of any third ECCM,
although the attendees expressed their interest in attending another ECCM
and several tentative offers of hosting the next meeting were put forward.
We certainly hope that something recognisably similar to the first two
ECCMs continues, though perhaps still more international in flavour.  To
judge from the papers at this conference, cognitive modelling in Europe is
in a comparatively healthy state.