batch-mode parallel implementations

[shams@maxwell.hrl.hac.com]

We have exploited the "epoch" training method for implementing back-prop on a 
2-D systolic array processor of Hughes [1,2].  There are two basic problems 
with this approach.  First, there are only a limited number of models that allow 
for epoch training (e.g. back-prop).  Second,  this type of parallelism is not 
useful during recall or classification cycle since there is only a single input 
pattern to be evaluated (unless the input data rate exceeds the processor 
throughput enabling the input data to be buffered for batch processing).  As the 
number of neurons used in real-world applications continue to increase,  there 
would be enough computation to keep all the processors busy without having to 
use epoch parallelism.
[1] S. Shams and K. W. Przytula, "Mapping of Neural Networks onto 
Programmable Parallel Machines," Proceedings of the Intern. Symp. on Circuits 
and Systems, New Orleans, LA, Vol. 4, pp. 2613-2617, 1990.

[2] S. Shams and K. W. Przytula. "Implementation of Multilayer Neural 
Networks on Parallel Programmable Digital Computers." In Parallel Algorithms 
and Architectures for DSP Applications. Ed. M. Bayoumi, Kluwer Academic 
Publishers, pp. 225-253, 1991.

Soheil Shams
Hughes Research Labs.