Why does the error rise in a SRN?

[Minh.Tue.Vo@cs.cmu.edu]

> Excerpts from connect: 2-Apr-92 Why does the error rise in ..
> mav at cs.uq.oz.au (886)

> I have been working with the Simple Recurrent Network (Elman style)
> and variants there of for some time. Something which seems to happen
> with surprising frequency is that the error will decrease for a period
> and then will start to increase again. I have seen the same phenomena
> using both root mean square and dprime error. It often occurs over


I have noticed the same phenomenon in my work on on-line gesture
recognition, so it doesn't seem to be task-dependent.  I could reduce
the effect somewhat by tweaking the learning rate and the momentum, but
I couldn't eliminate it completely.  TDNN doesn't seem to have that
problem.

Minh Tue Vo -- CMU