recurrent net bibliography

[thanasis kehagias]

a while ago i posted asking for information on recurrent nets.
i collected a lot of responses and compiled a bibliography, which i sent
to the list manager of connectionists. here is his response:

------------------------------------------------------------------
(List manager)
I've placed a copy of your message and bibliography (recurrent.bib) in the
"bibliographies" subdirectory of the connectionists directory that is
accessible via anonymous FTP (instructions included below).  I would suggest
you post a description of the bibliography to the list, telling people that it
is accessible via FTP and offering to mail it to people who cannot access it.
-------------------------------------------------------------------------------

        How to FTP Files from the CONNECTIONISTS Archive
        ------------------------------------------------

1.  Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address
    128.2.242.8).

2.  Login as user anonymous with password your username.

3.  'cd' directly to one of the following directories:
    /usr/connect/connectionists/archives
    /usr/connect/connectionists/bibliographies

4.  The archives and bibliographies directories are the ONLY ones you can
    access. You can't even find out whether any other directories exist.
    If you are using the 'cd' command you must cd DIRECTLY into one of these
    two directories. Access will be denied to any others, including their
    parent directory.

5.  The archives subdirectory contains back issues of the mailing list.
    Some bibliographies are in the bibliographies subdirectory.


Problems? - contact us at "Connectionists-Request at cs.cmu.edu".


      -------------------------------------------------------
(this is thanasis again):

so now you know there is a bibliography and how to access it. in the rest of
the message i include a small description of what is in the bibliography ...



****** THIS IS NOT A COMPLETE BIBLIOGRAPHY !!! ********

a while ago i posted a request for information on recurrent nets.
i got a lot of responses and i am posting back, as i had promised.
the bibliography follows at the end of the message.

a few words of explanation: i was not one hundred percent sure of
what i was looking for. having seen the responses, i think a fair
description of the list i am sending would be: "a personal study guide
for the time-space problem in connectionist networks" . what i mean is
the following: i would classify neural nets in two broad categories: (1)
static nets, where the output of nodes at time t is not fed back to
other nodes at time t+1, and (2) dynamic nets where the output of nodes
at time t is fed back to other nodes at time t+1. however, some of the
static nets attempt to capture temporal relationships, and they usually
do this by including time delayed inputs, this is quite commomn for the
speech researchers  (and i have included some references to such
STATIC work), as well as for the people doing the chaotic time series
problem (no references included here). for people with signal processing
background, this kind of static nets is similar  to FIR filters.

almost by definition, all the dynamic nets exhibit temporal
behavior. however, many of these dynamic nets are used in a static way.
e.g. Hopfield type nets are dynamic but they are most often used to
store static patterns. they are designed explicitly so they will settle
down to a steady state (no oscillations either). (but i must say
there are some few exceptions to this).

Boltzmann machines are similar: they are in constant thermal
motion, but we are basically interested in their steady state.

another clarification is important: the training phase involves
dynamic behavior in many cases even for static nets. as we vary the
weights, the behavior of the system changes in time. but again the basic
qualification about this "dynamic" behavior is that we are usually
interested in the equilibrium.

after all these qualifications, there are some truly dynamic nets,
(meaning they are designed so as to exhibit interesting non-equlibria)
analogous to the IIR filters in the signal processing paradigm ,
and my impression is that they are becoming more and more
popular. in my opinion they are the most interesting ones
and  maybe the future is in these truly dynamic nets.

having said all of the above, let me add that i include in the
following short bibliography almost everything that was sent to me.
needless to say, i have not read everything that i include in the list.
there is no attempt for completeness here, and omission of some work
should not be taken to mean that i consider this work inferior or
unimportant. in particular i need to make the following very clear: i
included very little ART type work, even if ART architectures
is an example of truly dynamic networks. the reason i did this is simply
that i am not familiar with this work.

the bibliography is very provisional. if you find it useful use
it, if not  useful, do not flame me.

i have a rather arbitrary grouping, with
sparse comments as to what each category has. maybe a more annotated
version will come later. all the more reason to send me suggestions
about additions or changes. and many many thanks to all the people who
send me contributions.

                   thanasis