Identity Mappings

[KINSELLAJ@vax1.nihel.ie]

                                                        John A. Kinsella

                                                        Mathematics Dept.,
                                                        University of Limerick,
                                                        Limerick,
                                                        IRELAND

                                                        KINSELLAJ at VAX1.NIHEL.IE

The strategy "identity mapping", namely training a feedforward network to
reproduce its input was (to the best of my knowledge) suggested by Geoffrey
Hinton and applied in a paper by J.L. Elman & D. Zipser "Learning the hidden
structure of speech".

It is not clear to me, however, that this approach can do more than aid in the
selection of the salient features of the data set. In other words what use is a
network which has been trained as an identity mapping on (say) a vision problem?

Certainly one can "strip off" the output layer & weights and by a simple piece
of linear algebra determine the appropriate weights to transform the hidden
layer states into output states corresponding to the salient features mentioned
above. It would appear, though, that this is almost as expensive a procedure
computationally as training the network as well as being numerically unstable
with respect to the subset of the training set selected for the purpose.

I would appreciate any comments on these remarks and in particular references to
relevant publised material,

                                                                John Kinsella