The best neural networks for classification

[mikewj@signal.dra.hmg.gb]

Dear Connectionists,

Many connectionist simulators are geared towards comparing one 
neural algorithm with another.  You can set different numbers of 
nodes, learning parameters etc. and do 20 runs (say) to get good 
statistical measurements on the performance of the algorithms on 
a dataset.

I am working on the European Commission funded Statlog project,
comparing a whole host of pattern classification techniques on some
large, dirty, and difficult industrial data sets; techniques being
tested include various neural net paradigms, as well as about 20
statistical and inductive inference methods.  Instead of the usual
rigorous performance figures useful for comparing different neural
nets, I have to produce the BEST NETWORK I CAN, and report performance
on training and test sets.

In order to do well on the test set, I need to hold back some of the
training data, and use this to evaluate the performance of networks of
different sizes and trained for different lengths of time, on the
remaining portion of the training data. Moreover, I would like to find
a simulator which uses faster training algorithms such as conjugate
gradients, which can cope with big datasets without having memory or
network nightmares, and which will do the hold-out cross-validation
itself, automatically.

My other options are to do this by hand (which is fine), but I see
greater benefits for the project, for "neural nets" as a standard
technique, and for industrial users unfamiliar with the trickeries of
data preparation and so on, if I can simply recommend a simulator
which will find the best network for an application, without a great
deal of intervention for evaluation, chopping of data and so on.

Thanks for your interest; any suggestions?

Mike Wynne-Jones.   mikewj at signal.dra.hmg.gb