NNs & TMs

[Richard Rohwer]

One way to emulate a Turing machine with a neural net is to hand-build
little subnets for NAND gates and FLIP FLOPS, and then wire them up to
do the job.  This observation led me to contemplate a project no student
seems to want to take up (and I don't want to do it anytime soon)-- so
I'll put it up for grabs in case anyone would be keen on it.  It should
be tedious but straightforward to write a compiler which turns C-code
(or your favorite language) into weight matrices.  Why bother? Well,
training algorithms are still in a pretty primitive state when it comes
to training nets to do complex temporal tasks; eg., parsing.  A weight
matrix compiler would at least provide an automatic way to initialize
weight matrices to do anything a conventional program can do, albeit
without using distributed representations in an interesting or efficient
way.  But perhaps something like minimizing an entropy measure from such
a starting point could lead to something interesting. 

Richard Rohwer                            JANET: rr at uk.ac.ed.eusip
Centre for Speech Technology Research      ARPA: rr%ed.eusip at nsfnet-relay.ac.uk
Edinburgh University                     BITNET: rr at eusip.ed.ac.uk,
80, South Bridge                                 rr%eusip.ed.UKACRL
Edinburgh  EH1 1HN,   Scotland             UUCP: ...!{seismo,decvax,ihnp4}
                                                          !mcvax!ukc!eusip!rr