Connectionists: a web essay to provoke discussion of sparse distributed representation as the key to biological intelligence

[Rod Rinkus]

(apologies for cross-posting)

I announce a new hyper-essay (at
http://www.sparsey.com/Sparsey_Hyperessay.html) describing a cortex-based
machine intelligence model, Sparsey, which I think will be of interest to
readers for several reasons.



First, Sparsey does storage, best-match retrieval, and belief update of
spatial or spatiotemporal inputs (hypotheses) with a number of operations
that remains fixed as the number of items stored in the database grows up
to a soft limit, N, that depends on model size (essentially the number of
weights, W).  This set of time performance properties has not been claimed
or shown for any other machine intelligence model, including the
world-leading Deep Learning (DL) models. This fixed time performance is not
constant time complexity, but empirical tests have shown that N scales well
in W.  This, in concert with the benefits of compositional organization of
information/knowledge afforded by deep hierarchies suggests that a model of
a fixed, reasonable W (e.g., several billion to several tens of billion
weights) may be able to operate without saturating over its lifetime and
explain the apparently huge capacity, speed (both during learning and
inference), and flexibility of biological, in particular, human, cognition.



Second, Sparsey's fixed-time storage (learning), best-match retrieval, and
belief update, capability depends crucially on the fact that information is
represented using sparse distributed representations (SDRs) in each Sparsey
module.  (N.b.: SDR is not the same concept as "sparse coding".)  It is
therefore not surprising that DL models do not have this fixed time
capability, since to my knowledge, SDR has not been used in any DL model.  On
the contrary. the long learning times of DL models, even using massive
machine parallelism (GPUs), is well known.  I suggest that the availability
of cheap massive machine parallelism may in fact be "garden-pathing"
researchers to continue pushing algorithms that may be fundamentally
different than the processes underlying biological intelligence.  I think
that the most important difference is the absence of SDR in DL models
(because of the fixed time performance SDR confers).   But there are other
obvious disconnects as well, e.g., the almost universal lack of mesoscale
architecture and function in DL models, in contrast to the brain's cortex,
for which there is substantial evidence of mesoscale structure and
function, and the use of gradient-based learning, which has long been
viewed as biologically implausible.



Third, it is noteworthy that the fixed time capabilities stated above have
not been claimed for any of the few other SDR-based models out there, in
particular, Numenta's HTM/Grok and Hecht-Nielsen's Cogent Confabulation.  This,
despite the obvious importance of fixed time performance for scaling to
"big data"-sized problems.



I hope readers view the essay and that its elaborations of the above points
and many other related ideas engenders a lively debate.



Sincerely,

Rod Rinkus

President, Neurithmic Systems


-- 
Gerard (Rod) Rinkus, PhD
President,
rod at neurithmicsystems dot com
Neurithmic Systems LLC <http://sparsey.com>
275 Grove Street, Suite 2-400
Newton, MA 02466
617-997-6272

Visiting Scientist, Lisman Lab
Volen Center for Complex Systems
Brandeis University, Waltham, MA
grinkus at brandeis dot edu
http://people.brandeis.edu/~grinkus/
<http://people.brandeis.edu/%7Egrinkus/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20160602/af6ee4ff/attachment.html>