new e-survey in NCS

[Arun Jagota]

New refereed e-publication
action editor: Yoshua Bengio      

W. Duch & N. Jankowski, Survey of Neural Transfer Functions  
Neural Computing Surveys 2, 163--212, 1999. 91 references.

http://www.icsi.berkeley.edu/~jagota/NCS

Abstract:                                    

The choice of transfer functions may strongly influence complexity
and performance of neural networks. Although sigmoidal transfer
functions are the most common, there is no /a priori/ reason why
models based on such functions should always provide optimal
decision borders. A large number of alternative transfer functions
have been described in the literature. A taxonomy of activation and
output functions is proposed, and advantages of various non-local
and local neural transfer functions are discussed. Several lesser
known types of transfer functions, and new combinations of
activation/output functions are described. Universal transfer
functions, parametrized to change from localized to delocalized
type, are of greatest interest. Other types of neural transfer
functions discussed here include functions with activations based on
non-Euclidean distance measures, bicentral functions, formed from
products or linear combinations of pairs of sigmoids, and extensions
of such functions making rotations of localized decision borders in
highly dimensional spaces practical. Nonlinear input preprocessing
techniques are briefly described, offering an alternative way to
change the shapes of decision borders.