Yet more on averaging

[Thomas M. Breuel]

I wrote, in response to a discussion of Michael Perrone's work:
|In general, averaging is clearly not optimal, nor even justifiable on
|theoretical grounds. [... some examples follow...]

Judging from some private mail that I have been receiving, some people
seem to have misunderstood my message.  I wasn't making a statement
about Michael's results per se, but about their application.

In particular, in the case of combining estimates of probabilities by
different "experts" for subsequent classification (e.g., in Michael's
OCR example), or in the case of combining expert "votes", using any
kind of linear combination is not justifiable in general on
theoretical grounds, and it is actually provably suboptimal in some
cases.

Now, such examples do violate some of the assumptions on which
Michael's results rely, so there is no contradiction.  My message was
only intended as a reminder that there are a number of important
problems in which the assumptions actually are violated, and in which
the approach of linear combinations reduces to a heuristic (one, I
might add, that often does work well in practice).

					Thomas.