Ph.D thesis on connectionist natural language processing

[Finn Dag Buoe]

The following doctoral thesis (and 3 of my related papers for COLING96, 
ECAI96, and ICSLP96) are available at the WWW page:

http://werner.ira.uka.de/ISL.speech.publications.html

--------------------------------------------------------------------------

FEASPAR - A FEATURE STRUCTURE PARSER LEARNING TO PARSE SPONTANEOUS SPEECH


                               (120 pages)

                              Finn Dag Buo

                               Ph.D thesis

                         University of Karlsruhe

                                Abstract 

Traditionally, automatic natural language parsing and translation have been 
performed with various symbolic approaches. Many of these have the advantage 
of a highly specific output formalism, allowing fine-grained parse analyses 
and, therefore, very precise translations. Within the last decade, statistical,
and connectionist techniques have been proposed to learn the parsing task in 
order to avoid the tedious manual modeling of grammar and malformation. How to 
learn a detailed output representation and how to learn to parse robustly even 
ill-formed input, has until now remained an open question.

This thesis provides an answer to this question by presenting a connectionist 
parser that needs a small corpus and a minimum of hand modeling, that learns, 
and that is robust towards spontaneous speech and speech recognizer effects. 
The parser delivers feature structure parses, and has a performance comparable 
to a good hand modeled unification based parser. 

The connectionist parser FeasPar consists of several neural networks and 
a Consistency Checking Search. The number of, architecture of, and other 
parameters of the neural networks are automatically derived from the training 
data. The search finds the combination of the neural net outputs that produces 
the most probable consistent analysis.

To demonstrate learnability and robustness, FeasPar is trained with 
transcribed sentences from the English Spontaneous Scheduling Task and 
evaluated for network, overall parse, and translation performance, with 
transcribed and speech data. The latter contains speech recognition errors. 
FeasPar requires only minor human effort and performs better or comparable 
to a good symbolic parser developed with a 2 year, human expert effort.
A key result is obtained by using speech data to evaluate the JANUS 
speech-to-speech translation system with different parsers. With FeasPar, 
acceptable translation performance is 60.5 %, versus 60.8 % with a GLR* parser.
FeasPar requires two weeks of human labor to prepare the lexicon and 600 
sentences  of training data, whereas the GLR* parser required significant 
human expert grammar modeling.

Presented in this thesis are the Chunk'n'Label Principle, showing how to divide 
the entire parsing tasks into several small tasks performed by neural networks, 
as well as the FeasPar architecture, and various methods for network performance 
improvement. Further, a knowledge analysis and two methods for improving the 
overall parsing performance are presented. Several evaluations and comparisons 
with a GLR* parser, producing exactly the same output formalism, illustrate 
FeasPar's advantages.


================================================================================
Finn Dag Buo
SAP AG
Germany
finn.buoe at sap-ag.de
================================================================================