TR available

[Jose del R. MILLAN]

The following Tech. Report is available. Requests should be sent to
		MILLAN at FIB.UPC.ES
________________________________________________________________________

		Learning by Back-Propagation:
	a Systolic Algorithm and its Transputer Implementation

		   Technical Report LSI-89-15

			Jose del R. MILLAN
		Dept. de Llenguatges i Sistemes Informatics
		Universitat Politecnica de Catalunya

			Pau BOFILL
		Dept. d'Arquitectura de Computadors
		Universitat Politecnica de Catalunya


ABSTRACT

In this paper we present a systolic algorithm for back-propagation, a 
supervised, iterative, gradient-descent, connectionist learning rule. The 
algorithm works on feedforward networks where connections can skip layers and 
fully exploits spatial and training parallelisms, which are inherent to 
back-propagation. Spatial parallelism arises during the propagation of activity 
---forward--- and error ---backward--- for a particular input-output pair. On 
the other hand, when this computation is carried out simultaneously for all 
input-output pairs, training parallelism is obtained. In the spatial dimension, 
a single systolic ring carries out sequentially the three main steps of the 
learnng rule ---forward, backward and weight increments update. Furthermore, the
same pattern of matrix delivery is used in both the forward and the backward 
passes. In this manner, the algorithm preserves the similarity of the forward 
and backward passes in the original model. The resulting systolic algorithm is 
dual with respect to the pattern of matrix delivery ---either columns or rows. 
Finally, an implementation of the systolic algorithm for the spatial dimension 
is derived, that uses a linear ring of Transputer processors.