connectionist NLP software, papers, web demos

[martym@cs.utexas.edu]

Dear Connectionists:

The following software package for connectionist natural language
processing and papers based on it are available from the UTCS Neural
Networks Research Group website http://www.cs.utexas.edu/users/nn.

Live demos of the software, including the systems described in the
papers, can be run remotely from the research description page
http://www.cs.utexas.edu/users/nn/pages/research/nlp.html.


=======================================================================
Software:
MIR: RAPID PROTOTYPING OF NEURAL NETWORKS FOR SENTENCE PROCESSING
http://www.cs.utexas.edu/users/nn/pages/software/abstracts.html#mir

The MIR software package has been designed for rapid prototyping of
typical architectures used in NLP research (such as SRN, RAAM, and SOM),
that depend heavily on (one or more) lexicons, and it can be easily
extended to handle other architectures as well.  It has been designed
to be simple to install, modify, and use, while retaining as much 
flexibility as possible.  To this end, the package is written in C
using the TCL/TK libraries.  The package includes a number of basic
commands and widgets, so that the user can quickly set up, train, and
test various architectures and visualize their dynamics.  High-level
scripts with training/testing data are also provided as examples of
running complete experiments with MIR.


=======================================================================
Papers:
=======================================================================
SARDSRN: A NEURAL NETWORK SHIFT-REDUCE PARSER

Marshall R. Mayberry, III, and Risto Miikkulainen.
Technical Report AI98-275, Department of Computer Sciences, The
University of Texas at Austin, 1998 (12 pages).

http://www.cs.utexas.edu/users/nn/pages/publications/abstracts.html#mayberry.sardsrn.ps.Z

Simple Recurrent Networks (SRNs) have been widely used in natural
language tasks. SARDSRN extends the SRN by explicitly representing the
input sequence in a SARDNET self-organizing map. The distributed SRN
component leads to good generalization and robust cognitive properties,
whereas the SARDNET map provides exact representations of the sentence
constituents. This combination allows SARDSRN to learn to parse
sentences with more complicated structure than can the SRN alone, and
suggests that the approach could scale up to realistic natural language.

=======================================================================
DISAMBIGUATION AND GRAMMAR AS EMERGENT SOFT CONSTRAINTS 

Risto Miikkulainen and Marshall R. Mayberry, III.
In B. J. MacWhinney (editor), Emergentist Approaches to Language.
Hillsdale, NJ: Erlbaum, in press (16 pages).

http://www.cs.utexas.edu/users/nn/pages/publications/abstracts.html#miikkulainen.emergent.ps.Z

When reading a sentence such as "The diplomat threw the ball in the
ballpark for the princess" our interpretation changes from a dance event
to baseball and back to dance. Such on-line disambiguation happens
automatically and appears to be based on dynamically combining the
strengths of association between the keywords and the two word
senses. Subsymbolic neural networks are very good at modeling such
behavior. They learn word meanings as soft constraints on
interpretation, and dynamically combine these constraints to form the
most likely interpretation. On the other hand, it is very difficult to
show how systematic language structures such as relative clauses could
be processed in such a system.  The network would only learn to
associate them to specific contexts and would not be able to process new
combinations of them. A closer look at understanding embedded clauses
shows that humans are not very systematic in processing grammatical
structures either. For example, "The girl who the boy who the girl who
lived next door blamed hit cried" is very difficult to understand,
whereas "The car that the man who the dog that had rabies bit drives is
in the garage" is not.  This difference emerges from the same semantic
constraints that are at work in the disambiguation task. In this chapter
we will show how the subsymbolic parser can be combined with high-level
control that allows the system to process novel combinations of relative
clauses systematically, while still being sensitive to the semantic
constraints.