Batch Learning and Parallel Implementation

[jim steck (ME]

Regarding Parallel implementations of Batch and online learning....
 
     S. Kollias and D. Anastassiou presented an interesting approximate
second order training algorithm using a Least Squares Estimation
Technique at IJCNN 1988  (IEEE Transactions on Circuits and Systems
vol 36 no. 8 ).
 
     This algorithm is interesting because it updates the weights with
each training pair, but performs the update using  information saved
from all previous training pairs.  The algorithm includes a parameter
called a forgetting factor which causes information from the previous
training pairs to slowly be discounted (or forgotten).  This is 
basically a type of learning somewhere inbetween "batch" learning and
"on line" learning.
 
     As an appoximate second order method, it is somewhat computationally
intensive; however, the method is easily and productively vectorized
on parallel architectures.
 
Jim Steck
Wichita State University