<div dir="ltr"><br>(apologies for cross-posting)<br><br><p style="margin:0in;font-family:Calibri;font-size:11pt">I announce a new
hyper-essay (at <a href="http://www.sparsey.com/Sparsey_Hyperessay.html" target="_blank">http://www.sparsey.com/Sparsey_Hyperessay.html</a>)
describing a cortex-based machine intelligence model, Sparsey, which I think
will be of interest to readers for several reasons.<span> </span></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> </p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">First, Sparsey does
storage, best-match retrieval, and belief update of spatial or spatiotemporal
inputs (hypotheses) with a number of operations that remains fixed as the
number of items stored in the database grows up to a soft limit, N, that
depends on model size (essentially the number of weights, W).<span> </span>This set of time performance properties has
not been claimed or shown for any other machine intelligence model, including
the world-leading Deep Learning (DL) models. This fixed time performance is not
constant time complexity, but empirical tests have shown that N scales well in
W.<span> </span>This, in concert with the benefits of
compositional organization of information/knowledge afforded by deep
hierarchies suggests that a model of a fixed, reasonable W (e.g., several
billion to several tens of billion weights) may be able to operate without
saturating over its lifetime and explain the apparently huge capacity, speed
(both during learning and inference), and flexibility of biological, in
particular, human, cognition.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> </p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">Second, Sparsey's
fixed-time storage (learning), best-match retrieval, and belief update,
capability depends crucially on the fact that information is represented using
sparse distributed representations (SDRs) in each Sparsey module.<span> </span>(N.b.: SDR is not the same concept as
"sparse coding".)<span> </span>It is
therefore not surprising that DL models do not have this fixed time capability,
since to my knowledge, SDR has not been used in any DL model.<span> </span>On the contrary. the long learning times of
DL models, even using massive machine parallelism (GPUs), is well known.<span> </span>I suggest that the availability of cheap
massive machine parallelism may in fact be "garden-pathing"
researchers to continue pushing algorithms that may be fundamentally different
than the processes underlying biological intelligence.<span> </span>I think that the most important difference is
the absence of SDR in DL models (because of the fixed time performance SDR confers).<span> </span>But
there are other obvious disconnects as well, e.g., the almost universal lack of
mesoscale architecture and function in DL models, in contrast to the brain's
cortex, for which there is substantial evidence of mesoscale structure and
function, and the use of gradient-based learning, which has long been viewed as
biologically implausible.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> </p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">Third, it is
noteworthy that the fixed time capabilities stated above have not been claimed
for any of the few other SDR-based models out there, in particular, Numenta's
HTM/Grok and Hecht-Nielsen's Cogent Confabulation.<span> </span>This, despite the obvious importance of fixed
time performance for scaling to "big data"-sized problems.<span> </span></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> </p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">I hope readers view
the essay and that its elaborations of the above points and many other related
ideas engenders a lively debate.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> </p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">Sincerely,</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">Rod Rinkus</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">President,
Neurithmic Systems</p><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Gerard (Rod) Rinkus, PhD<br>President,<br>rod at neurithmicsystems dot com<br><a href="http://sparsey.com" target="_blank">Neurithmic Systems LLC</a><br>275 Grove Street, Suite 2-400<br>Newton, MA 02466<br>617-997-6272<br><br>Visiting Scientist, Lisman Lab<br>Volen Center for Complex Systems<br>Brandeis University, Waltham, MA<br>grinkus at brandeis dot edu<br><a href="http://people.brandeis.edu/%7Egrinkus/" target="_blank">http://people.brandeis.edu/~grinkus/</a><a href="http://people.brandeis.edu/%7Egrinkus/" target="_blank"></a>
</div></div></div>
</div>