One to Many?

[Manoel Fernando Tenorio]

>> 
>> I disagree.  Its nothing to do with the architecture.  Its simply
>> that deterministic units cannot REPRESENT higher-order statistics
>> over the output units.
>> 
>> Geoff
>> 
Let me explain my point of view. I hope I understand you argument as posed by
your first email. Basically the stochastic machinery used in the BM capture
what would appear to be a covariance between the units. We have been try to
extend the BM to the continuous case, and that seem to be true. Now in the
case of deterministic units, given the proper transfer function (non-linearity)and the proper information (required number of links between units, one can
design networks to capture a variety of different features. Notice that 
deterministic units might not necessary be using a sigmoid function, but they
can used a series of more complex parameterized transfer functions, such as,
the GMDH algorithm (Molnar ICNN87), or spherically and polynomial graded
units (Hansen and Burr gte TR 87), or even the Multivariate Normal Distribution
units that we are experimenting with. Some problems, with special 
characteritics of the input pattern, allows the regular quasi-integrator to 
define a function similar to a Bayes classifier which optimizes MAP. 

Of course, if such retrictions on the input type are removed, the transfer
function has to adequately be modified and sometimes more links are also 
required to capture certain statistical characteristics. I really don't see
how that is only a function of whether the net is DET or STOCH, but  rather
of the unit transfer function and architectural characteristics.

If you modify the connection scheme in the BM, it would no longer capture the
same form of statistics, although the algorithm you remain the same (sort of
obvious, I guess). Similarly, if links are added between output units in DET
units,  interdependence would be more easily captured. Would could even imagine
schemes where output unit activation would go to a context unit, and then back
to the output unit (similar to JLElman CRL TR8801 UCSD), to capture temporal
covariances. Even simpler would be interunit links with a momentum term set
for about 1 cycle. 


--ft