Recurrent Linguistic Domain Papers?

[Richard Rohwer]

D. Servan-Schreiber writes, regarding simple recurrent networks...

> [...]
> We also explored the conditions under which the network can carry
> information about distant sequential contingencies across intervening
> elements to distant elements.  Such information is retained with relative
> ease if it is relevant at each intermediate step of a sequence; it tends to
> be lost when intervening elements do not depend on it.  [...]

The `Moving Targets' training algorithm (which is a bit of a pig in a
lot of ways) can deal with situations in which information from the
distant past is required in order to make a correct decision in the
present.  It can do this because error information is communicated
through time by additive tradeoffs in the cost function (which has
contributions from hidden nodes as well as target nodes), rather than by
the multiplicitive processes derived from the chain rule. 

I make no claims about biological plausibility.

References:
R. Rohwer, "The `Moving Targets' Training Algorithm" in Proc. EURASIP
Workshop on Neural Networks, Springer-Verlag Lecture Notes in Computer
Science No. 412. (1990).

R. Rohwer, "The `Moving Targets' Training Algorithm" to appear in Proc.
NIPS 1989

Richard Rohwer                            JANET: rr at uk.ac.ed.cstr
Centre for Speech Technology Research      ARPA: rr%ed.cstr at nsfnet-relay.ac.uk
Edinburgh University                     BITNET: rr at cstr.ed.ac.uk,
80, South Bridge                                 rr%cstr.ed.UKACRL
Edinburgh  EH1 1HN,   Scotland             UUCP: ...!{seismo,decvax,ihnp4}
                                                          !mcvax!ukc!cstr!rr
PHONE:  (44 or 0) (31) 225-8883 x280        FAX: (44 or 0) (31) 226-2730