Subtractive network design

[R Srikanth]

>
>
>     My point is that subtractive shemes are more likely to find
>     these global descriptions. These structures so to speek condense out of
>     the more complicated structures under the force of subtraction.
>
>     I would like to hear your opinion on this claim!
>
> A subtractive scheme can also lead to a network of about the right
> complexity, and you cite a couple of excellent studies that demonstrate
> this.  But I don't see why these should be better than additive methods
> (except for the problem noted above).  You suggest that a larger net can
> somehow form a good global description (presumably one that models a lot of
> the noise as well as the signal), and that the good stuff is more likely to
> be retained as the net is compressed.  I think it is equally likely that
> the global model will form some sort of description that blends signal and
> noise components in a very distributed manner, and that it is then hard to
> get rid of just the noisy parts by eliminating discrete chunks of network.
> That's my hunch, anyway -- maybe someone with more experience in
> subtractive methods can comment.
>

Also there is a question of over generalizations. A larger network say is
given a set of m points to learn a parabola, may end up learning a higher
order polynomial. Which is a case of over generalization leading to poor
performance. Of course the vice vorsa is also true.

The question posed here is do we need a first best fit or the most general
fit ?

Answer may be different for different problems. Thus we may be able to
generate opposite views in different problem spaces.

srikanth
--


srikanth at rex.cs.tulane.edu
Dept of Computer Science,
Tulane University,
New Orleans, La - 70118