No subject

[john kolen]

Higher order recurrent networks are recurrent networks with higher order
connections, (i[1]*i[2]*w[1,2] instead of i[1]*w[1]).  An example of a
high order recurent network is Pollack's sequential cascaded networks
which appear, I believe, in the latest issue of Machine Learning.  This
network can be described as two three-dimensional matrices, W and V, and
the following equations.

	O[t] = Sigmoid( (W . S[t]) . I[t])
	S[t+1]=Sigmoid( (V . S[t]) . I[t])

where I[t] is the input vector, O[t] is the output vector, and S[t] is the
state vector, each at time t.  ( . is inner product)

John Kolen