No subject

[Praveen Raina]

The following comparison between the backpropagation and
Kak algorithm for training feedforward networks will be
of interest to many.

We took 52 training samples each having 25 input neurons
and 3 output neurons.The training data taken was monthly
price index of a commodity for 60 months. Monthly prices 
were normalised and quantized into 3 bit binary sequence.
Each training sample represented prices taken over a period
of 8 months (8X3=24 input neurons + 1 neuron for bias).The
size of the learning window was fixed as 1 month.Binary 
values were used as the input for both BP and Kak algorithm.
For BP the learning rate was taken as 0.45 and momentum
equal to 0.55.

The training samples were trained on IBM RISC 6000 machine.
The training time for backpropagation was 4 minutes 5 seconds
and the total number of iterations was 6101.The training time
for the Kak algorithm was 5 seconds and the total number of
iterations was 875. Thus, for this example the learning advantage
in the Kak algorithm is 49. For larger examples the advantage
becomes even greater.
- Praveen Raina.