Connectionists: Deep Belief Nets (2006) / Neural History Compressor (1991) or Hierarchical Temporal Memory

[Brad Wyble]

>
>
>
> However, i am suggesting that the kinds of experiments we do, have all
> kinds of built in assumptions about learning in them to begin with, and
> that a great deal of machine learning, and NNs before that seems to assume
> fundamentally that these networks needed to learn from examples
> individually.  Many many years ago (sorry to keep dating myself), I pointed
> out in one of the first snowbird meetings that one needed to consider
> learning on individual as well as evolutionary time scales and that they
> were related in almost certainly a complex way.
>
>
>
Well I believe that you are certainly right that it's very complex, and
also that we build a lot of our theories into our experimental data. But
how else can one explain the ability to map concepts onto arbitrary
patterns of photons but through learning?  And how can this learned mapping
not be a fundamental part of our ability to deal with the visual world? The
ability to distinguish visual forms, very quickly, is what allows us to
deal with saccadic vision.

And what about reading?  Surely that constitutes a "real world" situation
in which learning is fundamental?

Incidentally, you might be really interested in project Prakash, which
restores sight to the congenitally blind, and thereby has the opportunity
to observe how quickly a visual system, that has been largely deprived of
input since birth, can adapt to the presence of vision.



>
>   Attention therefore provides a great example of a system that can be
> triggered by both hardwired (e.g. luminance and orientation defined
> stimuli), and acquired patterns (e.g. marine animals).
>
>
> I suspect they are not "acquired" in individual time, but rather in
> evolutionary time.  That is what I am saying - what form they exist in
> internally is another (and important obviously) question.
>
>
That's a very strong position, which is (if I understand you) essentially
saying that primitive visual forms are encoded in the DNA and expressed
through development in the visual system.  But if we critically depended on
such pre-existing forms, how could we ever learn to cope with technological
developments?  Or to learn a new language?  I think that you are
drastically underestimating the state space of vision,



Many years ago, I was visiting CNRS in Toulouse France, and Simon Thorpe
> had just finished one of the first "how fast can you recognize it" visual
> experiments.  After my talk (on cortical oscillations) he asked me if I
> could guess how fast a human could recognize the presence of an animal in a
> visual scene - I said under 200 Msecs - he was quite surprised that I
> guessed the answer  - I told him that it was one theta iteration.
>
>
Honestly, I think you just got lucky on that one.  There are plenty of
visual discriminations that require varying amounts of time from 200-500ms
or more.



> Point being, that humans can do this for animals they have never seen in
> the wild, or ever seen at all.
>
>
A bit unfair to call that de-novo performance, since most of us have seen a
sufficient variety of animals to allow us to accurately classify a novel
animal.   What is important to understand is that, with a few hours of
solid training, the ability to perceive/classify novel visual forms
increases dramatically.
(e.g. http://www.pnas.org/content/early/2012/12/19/1218438110.full.pdf)

If you still doubt, watch videos of world champion StarCraft players and
see if your visual system can keep up.

(e.g. http://www.youtube.com/watch?v=-yfMoIVTilo)




> call it a 'search image' - but I think, again, we attribute too much to
> learning, especially in the USA.
>
>
>
Well you may not be wrong there.  But I think that your perspective of the
NN field is a bit skewed by your experience.  There are quite a lot of us
who build networks that function "out of the box", and emphasize on-demand
function over learning.

-Brad

-- 
Brad Wyble
Assistant Professor
Psychology Department
Penn State University

http://wyblelab.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140211/88738df5/attachment.html>