The transposed weight matrix hassle

[Henrik Klagges]

There are some nasty things showing up if you want to fine-tune
a parallel architecture to algorithms such as backprop. E.g., you
either get the communications fast fro the forwar phase or the
backward phase - but if you want to use the same communication
flow for both, you have to transpose the weight matrices. This
is on the order of O(forget it). Has anybody cooked up an idea ?

Cheers, Henrik

MPCI at LLNL
IBM Research
U. of Munich