One to Many?

[Peter Marvit]

<This may be the first of a number of issues I've encountered so far.>

Most (nay all) of the simple connectionist networks which allow come sort
of pattern association are a one-to-one or many-to-one mapping.  For
example, given a set of inputs, light the "odd" or "even" output unit or
produce *the* past tense of a word given its root.  What simple
architecture exist for allowing a one-to-many mapping (e.g., given a, b or
c are allowed -- perhaps with equal frequency).

Why would one want this?  I'm trying to explore different mechanisms for
pluralizing nouns (extending some of Rumelhart and McClelland's work in
verb past tense generation.).  Unfortunately, some words (e.g., "fish")
have alternate forms.  So far, I've just thrown away the "less common"
plurals but it feels unsatisfactory in the general case.  I'm also using
the generalized delta rule with a standard back-propagation system;
eventually, I'll experiment with other architectures.

A few observations and random thoughts:

A one-to-many may actually be a one-to-one with given context.  Thus my
example may prefer "fish" when speaking about a "school of f." but prefer
"fishes" when talking about "three f."  This seems to unduly restrict
input, however.

One could think of an extension of this problem as "give all strings in
a list which match this substring."

One could construct a network with some units acting as "multiple answer"
flags which, if activated, would stochastically decide which "answer" to be
the output pattern.  This feels klugey and inelegant.  It also violates my
original request for a "simple architecture". Further, in the case of a
back-propagation network, it introduces the sole element of chance into an
otherwise completely deterministic system.  Is such a mixed mode system
realistic?

It may be that the simpler architectures are unsuitable for this.  If not,
what would be the simplest "mixed architecture" which would handle the problem?

What other applications would use a one-to-many?

-Peter Marvit
 HP Labs (part time)
 U.C. Berkeley (part time)