Batch Mode Parallel Implementations

[jim steck (ME]

     S. Kollias and D. Anastassiou presented an interesting approximate
second order training algorithm using a Least Squares Estimation
Technique at IJCNN 1988  (IEEE Transactions on Circuits and Systems
vol 36 no. 8 ).

     This algorithm is interesting because it updates the weights with
each training pair, but performs the update using  information saved
from all previous training pairs.  The algorithm includes a parameter
called a forgetting factor which causes information from the previous
training pairs to slowly be discounted (or forgotten).  This is 
essentially learning somewhere in between "batch learning" and
"on line learning".

     As an appoximate second order method, it is somewhat computationally
intensive; however, the method is easily and productively vectorized
on parallel architectures.

Jim Steck