No subject

[Yoshio Yamamoto]

One of my friends and I have been working on the applications of neural
networks in control problem independently.

After a little discussion we came across the following problem, which may be
interesting from a practical point of view. 

Suppose you have two continuous input units whose data are normalized between 
0 and 1, several hidden units, and one cotinuous output unit.
Also suppose the input given to the input unit A is totally unrelated with the
output; the input is a randomized number in [0,1].  The input unit B, on the
other hand, has a strong colleration with its corresponding output. 

Therefore what we need is a trained newtwork such that it shows no colleration
between the input A and the output.  This can be illustrated by an example
in which the input B is fixed, the input A varies at random in [0,1], and the
network suppresses the influence from the input A to minimum, ideally a 
constant output regardless of the values in the input A. In other words, we
want the output be fully independent of the input A. 
Then one obvious solution would be that all weights directed from the input A
to the next hidden layer converge to zeros or very small values through the
training process.

Why is this interesting?  This is useful in practical problem.  
Initially you don't know which input has colleration with the outputs and which
doesn't.  So you use all available inputs anyway.  If there is a nonsense 
input, then it should be identified so by a neural network and the influence
from the input should be automatically suppressed.

The best solution we have in mind is that if no colleration were identified, 
then the weights associated with the input will shrink to zero.


Is there any way to handle this problem? 

As a training tool we assume a backprop. 

Any suggestion will be greatly appreciated.


- Yoshio Yamamoto
  General Robotics And Sensory Perception Laboratory (GRASP)
  University of Pennsylvania