weight spaces

[alex shustorovich]

The following technical report seems to be relevant to this discussion:
______________________________________________________________________________

Reducing the Weight Space of a Net With Hidden Units to a Minimum Cone.

			   Alexander Shustorovich

		Image Electronics Center, Eastman Kodak Company
		  901 Elmgrove Road, Rochester NY 14653-5719

				ABSTRACT

   In his recent talk on the theory of Back-propagation at IJCNN-89, 
Dr. Hecht-Nielsen made an important observation that any single meaningful 
combination of weights can be represented in the net in a huge number of 
variants due to the permutations of hidden units. He remarked that if it 
were possible to find a cone in the weight space such that the whole space
is produced from this cone by permutations of axes corresponding to the 
permutations of the hidden units, it would greatly reduce the volume of 
space in which we have to organize the search for the solutions.

   In this paper such a cone is built. Besides the obvious benefits mentioned 
above, the same procedure enables the direct comparison of different solutions
and trajectories in the weight space, that is, the analysis and comparison of 
functions performed by individual hidden units.
______________________________________________________________________________

This paper was accepted for poster presentation at INNC-90-Paris in July
and it will appear in the proceedings. If you would like to have this TR now,
send your request to the author.

	Alexander Shustorovich,     email: sasha at alla.kodak.com