independence law

[James L. McClelland]

Several readers of connectionists have asked for clarification of the
independence law and/or pointers to publications.

The independence law (I call it that) is a finding in the literature
on the effect of context on perception of near-threshold stimuli
(Morton, Psychological Review, 1968).  It arises in studies in which
one manipulates contextual support for a particular response (e.g. by
providing a context such as 'I like coffee with cream and _____' vs
'The next word will be ______').  The '_____' represents the
near-threshold stimulus.  The context manipulation increases p = the
probability of choosing some response of interest (e.g. 'sugar')
relative to the neutral condition.

Now, the independence law is the finding that context exterts the same
effect on the variable log(p/(1-p)), independent of the exact nature
of the stimulus.  Of course, one needs a marginal stimulus to avoid p
= 1 or 0.  It had been claimed (Massaro, Cognitive Psychology, 1989)
that the independence law is inconsistent with symmetrically connected
networks, but this claim was based on simulations of the
(non-stochastic) interactive activation model of McClelland and
Rumelhart (Psych Review, 1981).  I found, though, (McClelland,
Cognitive Psychology, 1991) that in fact stochastic, symmetrically
connected networks can -- indeed, must -- adhere to the independence
law if they adhere to what I called the structural independence
constraint on architecture.  The constraint is roughly the following:

a) the network be structured so that there are three separate pools of units:
one for receiving and representing the stimulus input, one for
recieving and representing the context, and one for combining these
influences to represent the set of possible response alternatives
(e.g., words).

b) there can be no connections between any of the units in the
stimulus input part of the network and any of the units in the
context part of the network.

In my 91 paper the result was developed for Boltzmann machines using
localist representations of letters or words.  In recent work with
Javier Movellan (not yet ready for circulation) we are establishing
considerable generality to these results and relating the findings to
other literatures.  It turns out that what I called structural
independence in a stochastic symmetric network is tantamount to
adherence to an assumption called 'conditional independence' that is
used in pattern recognition in order to select the response with the
maximum a posteriori probability (MAP) given context and stimulus
information.  We will announce this paper when it is ready on
connectionists.