TR: Finding structure in time

[Jeff Elman]

Center for Research in Language Technical Report 8801/April

                "Finding structure in time"

                         Jeff Elman
                 Department of Linguistics
             University of California San Diego


     Time underlies many interesting human behaviors.  Thus,
the  question  of  how  to  represent  time in connectionist
models is very important.

     One approach is to represent  time  implicitly  by  its
effects  on  processing rather than explicitly (as in a spa-
tial representation).  The current report develops a  propo-
sal  along  these  lines  first  described by Michael Jordan
(1986) which involves the use of recurrent links in order to
provide  networks  with a dynamic memory.  In this approach,
hidden unit patterns are fed back to themselves; the  inter-
nal  representations which develop thus reflect task demands
in the context of prior internal states.

     A set of simulations  is  described  which  range  from
relatively simple proplems (temporal version of XOR) to dis-
covering syntactic/semantic features for words, to the prob-
lem  of  resolving  pronominal reference for sentences which
conform to Reinhart's (1983) C-command formulation.   In the
latter  case,  it is shown that a solution is possible which
does  not  require  the  symbol-processing  invoked  by   C-
command.   It  is  suggested  that  some aspects of language
behavior can be profitably viewed as  a  complex  sequential
behavior; the problem of discovering linguistic structure in
these cases is then the problem of discovering complex  tem-
poral structure.

----------------------------------------
Requests should be sent to J. Elman; Dept.  of  Linguistics,
C-008;  Univ.  of Calif., San Diego; La Jolla CA 92093-0108.
Email: elman at amos.ling.ucsd.edu