Squashing functions (continued)

[henrik@robots.ox.ac.uk]

The saturation problem ('the actviation function gets constant for large |x|')
can usually be solved by putting the derivative of the act. function into a 
table as well. You can then cheat a bit by not setting it to zero at large |x|.

Concerning memory requirements (eg, MasPar MP1). I don't see why I need 4 bytes
per table entry. According to the paper by Fahlman & Hoehfeld on limited pre-
cision, the quantization can be done with very few bits (less than 8 if tricks
are used). With interpolation you can get a pretty decent 16 bit act. value 
out of a 8bit wide table. 
Apart of that, seems to be quite complicated to put a nn on 16K processors ...
how do you do that ?
 
Cheers, henrik at robots.ox.ac.uk