Benchmark

[chrisley.pa@Xerox.COM]

By the way, Scott Fahlman's Aug 10th comment reminds me of another important
distinction in benchmarking:  learning rates vs. performance.  Since the bulk of
the tasks that have been used in benchmarking are deterministic (i.e.,
non-statistical), the performance comparison has been less interesting: any
network worth anything should achieve about the same performance, which is often
100%.  Since, in the statistical case, 0% error is generally impossible, the
performance of the networks becomes a much more interesting issue.  And in many
applications, the learning time is off-line, and therefore an irrelevant way to
judge the system.  

A good example where this might not be the case is, ironically, in our own
application of speech recognition.  Since none of the networks yet developed are
truly speaker independent, there must always be some re-calibration when you
want real-time, speaker independent recognition.  Thus, learning rates are
inmportant as well as performance.  Prof. Kohonen has it down to 10 minutes for
a new (Finnish) speaker, but that is not good enough for many applications.

-- Ron