Connectionists: from Leonid Litinskii, CONT RAS

[Litinskii]

Dear colleagues,

let us start from well-known conceptions. The one-level perceptron
with one output neuron is defined by n weights  w_i and by a
threshold  b.  At first, the output neuron creates an induced local field  

           v=sum (w_i x_i) -b.                     (1)

Then an activation function  f  transforms the induced local field  v
into an output signal y=f(v). 

We take an interest in topics that are related with the induced local
field  v only.  This field defines completely the form of the
separating surface. The ability of a perceptron to separate the given
set of points into two classes depends on the form of the separating
surface.    

For example, a hyperplane  is the separating surface corresponding to
the local field (1). Hyper-planes allow us to separate linearly
separable sets of the input points only. This is well-known as well as
the learning algorithms in this case.   
 
If we take the induced local field in the form

           v=sum(x_i -w_i)^2 -b,

the separating surface is  n-dimensional sphere whose center is in the
point W and whose radius is defined by the constant b. In this case
we deal with radial basic functions. The theory of these functions is
also well-known.   
 
It is found out that when the local field is only slightly differ from the form (1), 

                v = cos(X,W)-b,                 (2)

the separating surface is an angular sector, whose orientation and the
corner angle are defined by the values of the parameters W and  b.
Such separating surface allows us to solve the XOR-problem.  

We find out that a minor modification of the expression (2) allows us
to vary significantly the structure of the separating surface. In all
the cases the choosing of the parameters in Eq.(2) is realized
uniformly with the aid the gradient descent method. In other words,
with the aid of one-level perceptron and in the framework of the
uniform approach, it is possible to separate a set of the input points
into two classes. The points are distributed  nearly arbitrarily in
the space.       

We would like to know, if anybody studied this problem previously? 

It is impossible that nobody analyzed this problem. For example,
Wl. Duch in "K-Separability" (S.Kollias et al. (Eds.): ICANN 2006, Part I,
LNCS 4131, pp. 188-197, 2006) mentions the connection between the
form of the local field (2) and the solution of the XOR-problem. But
there were no further explanations.    

We'll be very grateful for any references on this topic.
  
Leonid Litinskii,
Center of Optical-Neural Technologies Russian Academy of Sciences

  

-- 
Best regards,
 Litinskii                          mailto:litin at iont.ru