Connectionists: Deep Belief Nets (2006) / Neural History Compressor (1991) or Hierarchical Temporal Memory

[Gary Marcus]

On Feb 10, 2014, at 4:51 PM, Brian J Mingus <brian.mingus at colorado.edu> wrote:

> That said, evolution is a blind designer. A human being can out-design billions of years of evolution in a few years with nice supercomputer and plenty of lab subjects. So, if your goal is to understand exactly what a human being is, you might study human development. But if your goal is to create something more sophisticated than a human without the annoyance of studying exactly how a human develops intelligence
[a reasonable goal, depending on your research program]

> , you might use deep networks with pretraining that automatically extract features that evolution baked in.
> 

the key question is whether extracting features is, in itself, enough to replicate (or even better) the blind handiwork of evolution. my own guess is “absolutely not”.  Evolution has a done a fine job with evo-crafting features — which deep networks might plausibly hope to match—  but that there’s a lot of highly-selected circuitry downstream that probably cannot readily be captured through the mere acquisition of hierarchies of features. 

On the left is a figure from Solari and Stoner’s magnificent cognitive consilience diagram, which I encourage all students of cortical neuroscience to contemplate (click to zoom in). On the right is a figure representing Google’s cat detector, a state of the art unsupervised learner, yet still no match for humans when it comes to invariance or in the use of top-down visual information. Is the one on the right genuinely a useful approximation of the one of the left?
 
In my own view there is an impedance mismatch between most current models and the intricacy of biological reality. 

Cheers,
Gary 

     

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140210/a630e443/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.png
Type: image/png
Size: 265368 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140210/a630e443/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-5.tiff
Type: image/tiff
Size: 37416 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140210/a630e443/attachment.tiff>