MATHEMATICAL PRINCIPLES OF REINFORCEMENT: BBS Call for Commentators

[Stevan Harnad]

Below is the abstract of a forthcoming target article by PETER KILLEEN
on MATHEMATICAL PRINCIPLES OF REINFORCEMENT, that has been accepted
for publication in Behavioral and Brain Sciences (BBS), an
international, interdisciplinary journal providing Open Peer Commentary
on important and controversial current research in the biobehavioral
and cognitive sciences. Commentators must be current BBS Associates or
nominated by a current BBS Associate. To be considered as a commentator
for this article, to suggest other appropriate commentators, or for
information about how to become a BBS Associate, please send email to:

harnad at clarity.princeton.edu  or harnad at pucc.bitnet        or write to:
BBS, 20 Nassau Street, #240, Princeton NJ 08542  [tel: 609-921-7771]

To help us put together a balanced list of commentators, please give some
indication of the aspects of the topic on which you would bring your
areas of expertise to bear if you were selected as a commentator. An
electronic draft of the full text is available for inspection by anonymous
ftp according to the instructions that follow after the abstract.
____________________________________________________________________

        MATHEMATICAL PRINCIPLES OF REINFORCEMENT

                Peter R. Killeen
                Department of Psychology
                University of Arizona
                Tempe, AZ 85281-1104

KEYWORDS: reinforcement, memory, coupling, contingency, contiguity, tuning 
curves, activation, schedules, trajectories, response rate, mathematical
models.

ABSTRACT: Effective conditioning requires a correlation between the
experimenter's definition of a response and an organism's, but an
animal's perception of its own behavior differs from ours. Various
definitions of the response are explored experimentally using the
slopes of learning curves to infer which comes closest to the
organism's definition. The resulting exponentially weighted moving
average provides a model of memory which grounds a quantitative theory
of reinforcement in which incentives excite behavior and focus the
excitement on the responses present in memory at the same time. The
correlation between the organism's memory and the behavior measured by
the experimenter is given by coupling coefficients derived for various
schedules of reinforcement. For simple schedules these coefficients can
be concatenated to predict the effects of complex schedules and can be
inserted into a generic model of arousal and temporal constraint to
predict response rates under any scheduling arrangement. According to
the theory, the decay of memory is response-indexed rather than
time-indexed. Incentives displace memory for the responses that occur
before them and may truncate the representation of the response that
brings them about. This contiguity-weighted correlation model bridges
opposing views of the reinforcement process and can be extended in a
straightforward way to the classical conditioning of stimuli. Placing
the short-term memory of behavior in so central a role provides a
behavioral account of a key cognitive process.
--------------------------------------------------------------
To help you decide whether you would be an appropriate commentator for
this article, an electronic draft is retrievable by anonymous ftp from
princeton.edu according to the instructions below (the filename is
bbs.killeen). Please do not prepare a commentary on this draft. Just
let us know, after having inspected it, what relevant expertise you
feel you would bring to bear on what aspect of the article.
-------------------------------------------------------------
   To retrieve a file by ftp from a Unix/Internet site, type either:
ftp princeton.edu
   or
ftp 128.112.128.1
   When you are asked for your login, type:
anonymous
   Enter password as per instructions (make sure to include the specified @),
   and then change directories with:
cd /pub/harnad/BBS
   To show the available files, type:
ls
   Next, retrieve the file you want with (for example):
get bbs.killeen
   When you have the file(s) you want, type:
quit
   In case of doubt or difficulty, consult your system manager.

----------
Where the above procedures are not available (e.g. from Bitnet or other
networks), there are two fileservers:
ftpmail at decwrl.dec.com
       and
bitftp at pucc.bitnet
that will do the transfer for you. To one or the
other of them, send the following one line message:

help

for instructions (which will be similar to the above, but will be in
the form of a series of lines in an email message that ftpmail or
bitftp will then execute for you).
-------------------------------------------------------------