Connectionists: Brain-like computing fanfare and big data fanfare

[Anders Lansner]

[Resending this to the intended thread]

 

Dear John and all,

 

I was not aware until this morning that my simple announcement of the
workshop on "Progress in Brain-Like Computing" at KTH Royal Institute of
Technology in Stockholm next week had stirred up such a vivid discussion on
the list. I was on a conference trip to Singapore with only occasional web
access and got back only yesterday. Allow me some comments and reflections.

 

I agree with some of the original points made by John Weng on that we need
brain-scale theories in order to make real progress in brain-like computing
and what the focus should be. Indeed, I think we see some maybe vague
contours of partial theories that wait to be integrated to a more complete
understanding.

 

Since the terminology of "brain-like" was criticized from different
perspectives, allow me some motivation why we use this term. We could have
stated "neuromorphic" but in my opinion this term leads the thoughts a bit
too much towards microscopic and microcircuit levels. After all real brains
not only have very many neurons and synapses but also a very complex
structure in terms of specialized neural populations and projections
connecting them. We have today chips and clusters that are able to simulate
with reasonable throughput such multi-neural-network structures (if not too
complex components .) so we can at least computationally handle this level,
rather than staying with small simple networks. Personally I think that to
understand principles of brain function we need to avoid a lot but not all
of the complexity we see at the single neuron and synapse levels. I also
prefer the term "brain-like" rather than "brain-inspired" since the former
defines the goal of building computational structures that are like brains
and not just to start there and then perhaps quickly, in all our
inspiration, diverge away from mimicking the essential aspects of real
brains.

 

It is interesting to note that the subject of the discussion quickly
deviated from the main content of our workshop which has to do with
designing and eventually building brain-like computational architectures in
silicon - or some more exotic substrate. Such research has been going on for
long time and is now seeing increasing efforts. It can obviously be argued
whether this is still premature or if it is now finally the right time to
boost such efforts. Despite the fact that our knowledge about the brain is
still not complete .

 

What also strikes me when I read this discussion is that we are still quite
a divided and diverse community with minor consensus. There are many who
think we are many decades away from doing the above, many who study abstract
computational "deep learning" network models for classification and
prediction without bothering much about the biology, many who study
experimentally or model brain circuits without focusing much on what
functions they perform, and many who design hardware without knowing exactly
what features to include, etc. 

 

But I am optimistic! Perhaps, in the near future, these efforts will combine
synergistically and the pieces of the puzzle will start falling in place,
triggering a series of real breakthroughs in our understanding of how our
brain works. To identify at what point in time and what stage in brain
science this will happen is indeed critical. Then, those who have the best
understanding of how to design the hardware appropriate for executing in
real time or faster this integrated set of brain-like algorithms in a
low-power way will be in an excellent position for exploiting such progress
in many important applications - hopefully beneficial for mankind!

 

This is some of the background for organizing the event I announced, which
will hopefully contribute something to the further discussion on these very
important topics.

 

/Anders La

 

From: Connectionists [mailto:connectionists-bounces at mailman.srv.cs.cmu.edu]
On Behalf Of Juyang Weng
Sent: den 26 januari 2014 07:27
To: connectionists at mailman.srv.cs.cmu.edu
Subject: Re: Connectionists: Brain-like computing fanfare and big data
fanfare

 

I enjoyed many views of you, including those of Jim Bower, Richard
Loosemore, Ali Minai, and Thomas Trappenberg.  Let me give a humble
suggestion, when you hear a detailed view that looks "polarizing", do not
get discouraged as your emotion is mistreating you.  Find out whether the
detail fits many brain functions.  

Big data and brain-like computing will fail if such a project is without the
guidance of a brain-scale model. 

Note: vague statements are not very useful here as everybody can give many.
A brain-scale model must be very detailed computationally.  The more
detailed, the more brain-scale functions it covers, and the fewer mechanisms
it uses, the better. 

For example, the Spaun model claimed to be " the world's largest functional
brain model".  With great respect, I congratulate its appearance in Science
2012.  But, unfortunately, Science editors are not in a position to judge
how close a brain model is.  I understand that no model of the brain is
perfect and every model is an approximation of the nature.  From my
brain-scale model, I think that a minimal requirement for a reviewer on a
brain model must have a formal training (e.g., a 3-credit course and pass
all its exams) in all the following: 

(1) computer vision,
(2) artificial intelligence,
(3) automata theory and computational complexity, 
(4) electrical engineering, such as signals and systems, control,
conventional neural networks,
(5) biology,
(6) neuroscience,
(7) cognitive science, such as learning and memory, human vision systems,
and developmental psychology,
(8) mathematics, such as linear algebra, probability, statistics, and
optimization theory.
If you do not have some of the above, take such courses as soon as possible.
BMI summer courses 2014 will offer some. 

If you have taken all the above courses, you will know that the Spaun model
is grossly wrong (and, with respect, the deep learning net of Geoffery
Hinton for the same reason).  

Why?  I just give the first mechanism that every brain must have and thus
every brain model must have:
learning and recognizing unknown objects FROM unknown cluttered backgrounds
and producing desired behaviors

Note: not just recognizing but learning; not a single object in a clean
background that Spaun demonstrated but also simultaneous multiple objects in
a cluttered backgrounds.  No objects can be pre-segmented from the cluttered
background during learning.  That is how a baby learns. 

None of the tasks that Spaun did includes cluttered background, let along
learning directly from cluttered scenes.  

Attention is the first basic mechanism of the brain learned from the baby
time, not recognizing a pattern in a clean background. 

Autonomously learning attention is the single most important mechanism for
Big Data and Brain-Like Computing!  
How?  Read How the Brain-Mind Works: A Two-Page Introduction to a Theory
<http://www.brain-mind-magazine.org/read.php?file=BMM-V2-N2-a1-HowBrainMind-
a.pdf>  banner






-John



On 1/24/14 9:03 PM, Thomas Trappenberg wrote:

Thanks John for starting a discussion ... I think we need some. What I liked
most about your original post was asking about "What are the underlying
principles?" Let's make a list.

Of course, there are so many levels of organizations and mechanisms in the
brain, that we might speak about different things; but getting different
views would be fun and I think very useful (without the need to offer the
only and ultimate).

Cheers, Thomas Trappenberg



PS: John, I thought you started a good discussion before, but I got
discouraged by your polarizing views. I think a lot of us can relate to you,
but lhow about letting others come forward now?

 

On Fri, Jan 24, 2014 at 9:02 PM, Ivan Raikov <ivan.g.raikov at gmail.com>
wrote:

 

I think perhaps the objection to the Big Data approach is that it is applied
to the exclusion of all other modelling approaches. While it is true that
complete and detailed understanding of  neurophysiology and anatomy is at
the heart of neuroscience, a lot can be learned about signal propagation in
excitable branching structures using statistical physics, and a lot can be
learned about information representation and transmission in the brain using
mathematical theories about distributed communicating processes. As these
modelling approaches have been successfully used in various areas of
science, wouldn't you agree that they can also be used to understand at
least some of the fundamental properties of brain structures and processes? 

  -Ivan Raikov

 

On Sat, Jan 25, 2014 at 8:31 AM, james bower <bower at uthscsa.edu> wrote:

[snip] 

An enormous amount of engineering and neuroscience continues to think that
the feedforward pathway is from the sensors to the inside - rather than
seeing this as the actual feedback loop.  Might to some sound like a
semantic quibble,  but I assure you it is not.

 

If you believe as I do, that the brain solves very hard problems, in very
sophisticated ways, that involve, in some sense the construction of complex
models about the world and how it operates in the world, and that those
models are manifest in the complex architecture of the brain - then
simplified solutions are missing the point.

 

What that means inevitably, in my view, is that the only way we will ever
understand what brain-like is, is to pay tremendous attention experimentally
and in our models to the actual detailed anatomy and physiology of the
brains circuits and cells.

 

 





-- 
--
Juyang (John) Weng, Professor
Department of Computer Science and Engineering
MSU Cognitive Science Program and MSU Neuroscience Program
428 S Shaw Ln Rm 3115
Michigan State University
East Lansing, MI 48824 USA
Tel: 517-353-4388
Fax: 517-432-1061
Email: weng at cse.msu.edu
URL: http://www.cse.msu.edu/~weng/
----------------------------------------------
 


---
Detta epostmeddelande innehåller inget virus eller annan skadlig kod för avast! antivirus är aktivt.
http://www.avast.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140126/56452e6a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2734 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140126/56452e6a/attachment.jpg>