No subject

[Erkki Oja]

RE: PCA in neural networks

Dear Terry + connectionists:

I will continue shortly the discussion about PCA, Hebbian rule
and deflation. I would also like to point out the "Weighted Subspace
algorithm" which Luis Almeida already mentioned. A more comprehensive
reference is given at the end of this note. While in the GHA and
SGA methods, the j-th weight vector depends on all the others up to
index j, the Weighted Subspace algorithm is homogeneous in the 
sense that the j-th weight vector depends on all the others. I would
say that the latter type is not deflation. 

There is an extension, mentioned in ref. (2) below, which is totally
symmetrical, but there is a nonlinearity in the "feedback" term
of the algorithm. This is sufficient to drive the vectors to the
true eigenvectors. This was pointed out to me first by L. Xu.

The references:


1. E. Oja, H. Ogawa and J. Wangviwattana: "Principal component
analysis by homogeneous neural networks, Part I: The weighted
subspace criterion". IEICE Trans. Inf. and Systems, vol. E75-D,
no. 3, May 1992, pp. 366 - 375.

2. E. Oja, H. Ogawa and J. Wangviwattana: "Principal component
analysis by homogeneous neural networks, Part II: Analysis and 
extensions of the learning algorithms. Same as above, pp. 376 - 
382. 


Regards, Erkki Oja (Erkki.Oja at hut.fi)