Papers available on connectionist NLP, neuro-evolution/Othello

[Risto Miikkulainen]

The following papers on 
 - processing complex sentences, 
 - disambiguation in distributed parsing networks,
 - learning German verb inflections, and
 - evolving networks to play Othello 
are available by anonymous ftp from our archive site at
cs.utexas.edu:pub/neural-nets/papers.

-- Risto Miikkulainen

-------------------------------------------------------------------------
miikkulainen.subsymbolic-caseroles.ps.Z (21 pages)

SUBSYMBOLIC CASE-ROLE ANALYSIS OF SENTENCES WITH EMBEDDED CLAUSES

Risto Miikkulainen
Department of Computer Sciences, The University of Texas at Austin.

Technical Report AI93-202, July 1993.

A distributed neural network model called SPEC for processing sentences
with recursive relative clauses is described. The model is based on
separating the tasks of segmenting the input word sequence into clauses,
forming the case-role representations, and keeping track of the
recursive embeddings into different modules. The system needs to be
trained only with the basic sentence constructs, and it generalizes not
only to new instances of familiar relative clause structures, but to
novel structures as well. SPEC exhibits plausible memory degradation as
the depth of the center embeddings increases, its memory is primed by
earlier constituents, and its performance is aided by semantic
constraints between the constituents.  The ability to process structure
is largely due to a central executive network that monitors and controls
the execution of the entire system. This way, in contrast to earlier
subsymbolic systems, parsing is modeled as a controlled high-level
process rather than one based on automatic reflex responses.


-------------------------------------------------------------------------
mayberry.disambiguation.ps.Z (10 pages)

LEXICAL DISAMBIGUATION BASED ON DISTRIBUTED REPRESENTATIONS 
OF CONTEXT FREQUENCY

Marshall R. Mayberry, III, and Risto Miikkulainen
Department of Computer Sciences, The University of Texas at Austin.

Technical Report AI94-217, February 1994.

A model for lexical disambiguation is presented that is based on
combining the frequencies of past contexts of ambiguous words.  The
frequences are encoded in the word representations and define the words'
semantics. A Simple Recurrent Network (SRN) parser combines the context
frequences one word at a time, always producing the most likely
interpretation of the current sentence at its output. This
disambiguation process is most striking when the interpretation involves
semantic flipping, that is, an alternation between two opposing meanings
as more words are read in. The sense of throwing a ball alternates
between dance and baseball as indicators such as the agent, location,
and recipient are input. The SRN parser demonstrates how the context
frequencies are dynamically combined to determine the interpretation of
such sentences. We hypothesize that other aspects of ambiguity
resolution are based on similar mechanisms are well, and can be
naturally approached from the distributed connectionist viewpoint.


-------------------------------------------------------------------------
westermann.inflections.ps.Z (9 pages)

VERB INFLECTIONS IN GERMAN CHILD LANGUAGE: A CONNECTIONIST ACCOUNT

Gert Westermann(1) and Risto Miikkulainen(2)
(1) Department of Computer Science, Technical University of Braunschweig.
(2) Department of Computer Sciences, The University of Texas at Austin.

Technical Report AI94-216, February 1994.

The emerging function of verb inflections in German language acquisition
is modeled with a connectionist network. A network that is initially
presented only with a semantic representation of sentences uses the
inflectional verb ending -t to mark those sentences that are low in
transitivity, whereas all other verb endings occur randomly.  This
behavior matches an early stage in German language acquisition where
verb endings encode a similar semantic rather than a grammatical
function.  When information about the surface structure of the sentences
is added to the input data, the network learns to use the correct verb
inflections in a process very similar to children's learning.  This
second phase is facilitated by the semantic phase, suggesting that there
is no shift from semantic to grammatical encoding, but rather an
extension of the initial semantic encoding to include grammatical
information.  This can be seen as evidence for the strong version of the
functionalist hypothesis of language acquisition.


-------------------------------------------------------------------------
moriarty.othello.ps.Z (6 pages)

EVOLVING COMPLEX OTHELLO STRATEGIES USING MARKER-BASED GENETIC
ENCODING OF NEURAL NETWORKS

David E. Moriarty and Risto Miikkulainen
Department of Computer Sciences, The University of Texas at Austin,
Austin, TX 78712.

Technical Report AI93-206, September 1993.

A system based on artificial evolution of neural networks for developing
new game playing strategies is presented.  The system uses marker-based
genes to encode nodes in a neural network.  The game-playing networks
were forced to evolve sophisticated strategies in Othello to compete
first with a random mover and then with an alpha-beta search program.
Without any direction, the networks discovered first the standard
positional strategy, and subsequently the mobility strategy, an advanced
strategy rarely seen outside of tournaments. The latter discovery
demonstrates how evolution can develop novel solutions by turning an
initial disadvantage into an advantage in a changed environment.

[ see also moriarty.focus.ps.Z: "Evolving Neural Networks to Focus
Minimax Search" ]