analog noise

[Mike Casey]

Regarding the RNN+noise implying only finite state machine power result, I
completely agree that their result is a nice extension of the corollary
that I proved (which I recently found to be a conjecture/assumption of
Turing in his 1936 paper), but they failed to mention that I had already
proved something very similar.  It seemed that a discussion of the type
posted here belonged in the paper. 

> In addition we have also relaxed the definition of analog noise in 
> our model to include also Gaussian noise etc, but this is perhaps
> a less essential point.

This is still incorrect.  Their "clipped" Gaussian noise is a special
case of bounded noise with arbitrary distribution (Bowen's pseudo-orbit
formalism), so there's no sense in which they "relaxed" the definition of
analog noise.  If the state space is bounded, and the noise is clipped to
keep the state of the system in the state space, then there is no loss of
generality in using bounded noise.  Furthermore, none of their results
depend on the noise being unbounded (so even if it were unbounded, it
wouldn't lead to anything interesting).  Finally, in section 4 of their
paper where they concretely discuss RNNs performing computations, they
assume that the noise is bounded and that the computation is done with
perfect reliability (which were precisely the assumptions that I used
which they have spent so much time discrediting in other parts of the
paper).

I sincerely apologize for taking up more bandwidth with this discussion. 
I hope that it is of some interest to the community.  Any further
discussion on my part will take place off-line. 


Best regards,
Mike


********************************************************************
Mike Casey
Volen Center for Complex Systems Studies
Brandeis University
Waltham, MA  02254
email:  mcasey at volen.brandeis.edu
http://eliza.cc.brandeis.edu/people/mcasey
(617) 736-3144 (voice)  (617) 736-3142 (fax)