<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
..shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
..MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=white lang=SV link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='color:windowtext'>[Resending this to the intended thread]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:windowtext'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:windowtext'>Dear John and all,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I was not aware until this morning that my simple announcement of the workshop on ”Progress in Brain-Like Computing” at KTH Royal Institute of Technology in Stockholm next week had stirred up such a vivid discussion on the list. I was on a conference trip to Singapore with only occasional web access and got back only yesterday. Allow me some comments and reflections.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I agree with some of the original points made by John Weng on that we need brain-scale theories in order to make real progress in brain-like computing and what the focus should be. Indeed, I think we see some maybe vague contours of partial theories that wait to be integrated to a more complete understanding.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Since the terminology of “brain-like” was criticized from different perspectives, allow me some motivation why we use this term. We could have stated “neuromorphic” but in my opinion this term leads the thoughts a bit too much towards microscopic and microcircuit levels. After all real brains not only have very many neurons and synapses but also a very complex structure in terms of specialized neural populations and projections connecting them. We have today chips and clusters that are able to simulate with reasonable throughput such multi-neural-network structures (if not too complex components …) so we can at least computationally handle this level, rather than staying with small simple networks. Personally I think that to understand principles of brain function we need to avoid a lot but not all of the complexity we see at the single neuron and synapse levels. I also prefer the term “brain-like” rather than “brain-inspired” since the former defines the goal of building computational structures that are like brains and not just to start there and then perhaps quickly, in all our inspiration, diverge away from mimicking the essential aspects of real brains.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>It is interesting to note that the subject of the discussion quickly deviated from the main content of our workshop which has to do with designing and eventually building brain-like computational architectures in silicon – or some more exotic substrate. Such research has been going on for long time and is now seeing increasing efforts. It can obviously be argued whether this is still premature or if it is now finally the right time to boost such efforts. Despite the fact that our knowledge about the brain is still not complete …<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>What also strikes me when I read this discussion is that we are still quite a divided and diverse community with minor consensus. There are many who think we are many decades away from doing the above, many who study abstract computational “deep learning” network models for classification and prediction without bothering much about the biology, many who study experimentally or model brain circuits without focusing much on what functions they perform, and many who design hardware without knowing exactly what features to include, etc. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>But I am optimistic! Perhaps, in the near future, these efforts will combine synergistically and the pieces of the puzzle will start falling in place, triggering a series of real breakthroughs in our understanding of how our brain works. To identify at what point in time and what stage in brain science this will happen is indeed critical. Then, those who have the best understanding of how to design the hardware appropriate for executing in real time or faster this integrated set of brain-like algorithms in a low-power way will be in an excellent position for exploiting such progress in many important applications – hopefully beneficial for mankind!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>This is some of the background for organizing the event I announced, which will hopefully contribute something to the further discussion on these very important topics.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>/Anders La<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'> Connectionists [mailto:connectionists-bounces@mailman.srv.cs.cmu.edu] <b>On Behalf Of </b>Juyang Weng<br><b>Sent:</b> den 26 januari 2014 07:27<br><b>To:</b> connectionists@mailman.srv.cs.cmu.edu<br><b>Subject:</b> Re: Connectionists: Brain-like computing fanfare and big data fanfare<o:p></o:p></span></p></div></div><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:12.0pt'>I enjoyed many views of you, including those of Jim Bower, Richard Loosemore, Ali Minai, and Thomas Trappenberg. Let me give a humble suggestion, when you hear a detailed view that looks "polarizing", do not get discouraged as your emotion is mistreating you. Find out whether the detail fits many brain functions. <br><br>Big data and brain-like computing will fail if such a project is without the guidance of a brain-scale model. <br><br>Note: vague statements are not very useful here as everybody can give many. A brain-scale model must be very detailed computationally. The more detailed, the more brain-scale functions it covers, and the fewer mechanisms it uses, the better. <br><br>For example, the Spaun model claimed to be " the world's largest functional brain model". With great respect, I congratulate its appearance in Science 2012. But, unfortunately, Science editors are not in a position to judge how close a brain model is. I understand that no model of the brain is perfect and every model is an approximation of the nature. From my brain-scale model, I think that a minimal requirement for a reviewer on a brain model must have a formal training (e.g., a 3-credit course and pass all its exams) in all the following: <br><br>(1) computer vision,<br>(2) artificial intelligence,<br>(3) automata theory and computational complexity, <br>(4) electrical engineering, such as signals and systems, control, conventional neural networks,<br>(5) biology,<br>(6) neuroscience,<br>(7) cognitive science, such as learning and memory, human vision systems, and developmental psychology,<br>(8) mathematics, such as linear algebra, probability, statistics, and optimization theory.<br>If you do not have some of the above, take such courses as soon as possible. BMI summer courses 2014 will offer some. <br><br>If you have taken all the above courses, you will know that the Spaun model is grossly wrong (and, with respect, the deep learning net of Geoffery Hinton for the same reason). <br><br>Why? I just give the first mechanism that every brain must have and thus every brain model must have:<br>learning and recognizing unknown objects FROM unknown cluttered backgrounds and producing desired behaviors<br><br>Note: not just recognizing but learning; not a single object in a clean background that Spaun demonstrated but also simultaneous multiple objects in a cluttered backgrounds. No objects can be pre-segmented from the cluttered background during learning. That is how a baby learns. <br><br>None of the tasks that Spaun did includes cluttered background, let along learning directly from cluttered scenes. <br><br>Attention is the first basic mechanism of the brain learned from the baby time, not recognizing a pattern in a clean background. <br><br>Autonomously learning attention is the single most important mechanism for Big Data and Brain-Like Computing! <br>How? Read <a href="http://www.brain-mind-magazine.org/read.php?file=BMM-V2-N2-a1-HowBrainMind-a.pdf">How the Brain-Mind Works: A Two-Page Introduction to a Theory</a> <img border=0 width=100 height=100 id="_x0000_i1025" src="cid:image001.jpg@01CF1AEF.C3FD5860" alt=banner><br><br><br><br><br><br><br>-John<br><br><o:p></o:p></p><div><p class=MsoNormal>On 1/24/14 9:03 PM, Thomas Trappenberg wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><div><div><p class=MsoNormal>Thanks John for starting a discussion ... I think we need some. What I liked most about your original post was asking about "What are the underlying principles?" Let's make a list.<o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt'>Of course, there are so many levels of organizations and mechanisms in the brain, that we might speak about different things; but getting different views would be fun and I think very useful (without the need to offer the only and ultimate).<o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt'>Cheers, Thomas Trappenberg<br><br><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt'>PS: John, I thought you started a good discussion before, but I got discouraged by your polarizing views. I think a lot of us can relate to you, but lhow about letting others come forward now?<o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On Fri, Jan 24, 2014 at 9:02 PM, Ivan Raikov <<a href="mailto:ivan.g.raikov@gmail.com" target="_blank">ivan.g.raikov@gmail.com</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal style='margin-bottom:12.0pt'>I think perhaps the objection to the Big Data approach is that it is applied to the exclusion of all other modelling approaches. While it is true that complete and detailed understanding of neurophysiology and anatomy is at the heart of neuroscience, a lot can be learned about signal propagation in excitable branching structures using statistical physics, and a lot can be learned about information representation and transmission in the brain using mathematical theories about distributed communicating processes. As these modelling approaches have been successfully used in various areas of science, wouldn't you agree that they can also be used to understand at least some of the fundamental properties of brain structures and processes? <o:p></o:p></p></div><div><p class=MsoNormal> -Ivan Raikov<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Sat, Jan 25, 2014 at 8:31 AM, james bower <<a href="mailto:bower@uthscsa.edu" target="_blank">bower@uthscsa.edu</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal>[snip] <o:p></o:p></p></div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><p class=MsoNormal>An enormous amount of engineering and neuroscience continues to think that the feedforward pathway is from the sensors to the inside - rather than seeing this as the actual feedback loop. Might to some sound like a semantic quibble, but I assure you it is not.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>If you believe as I do, that the brain solves very hard problems, in very sophisticated ways, that involve, in some sense the construction of complex models about the world and how it operates in the world, and that those models are manifest in the complex architecture of the brain - then simplified solutions are missing the point.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>What that means inevitably, in my view, is that the only way we will ever understand what brain-like is, is to pay tremendous attention experimentally and in our models to the actual detailed anatomy and physiology of the brains circuits and cells.<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></blockquote></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></blockquote><p class=MsoNormal><br><br><o:p></o:p></p><pre>-- <o:p></o:p></pre><pre>--<o:p></o:p></pre><pre>Juyang (John) Weng, Professor<o:p></o:p></pre><pre>Department of Computer Science and Engineering<o:p></o:p></pre><pre>MSU Cognitive Science Program and MSU Neuroscience Program<o:p></o:p></pre><pre>428 S Shaw Ln Rm 3115<o:p></o:p></pre><pre>Michigan State University<o:p></o:p></pre><pre>East Lansing, MI 48824 USA<o:p></o:p></pre><pre>Tel: 517-353-4388<o:p></o:p></pre><pre>Fax: 517-432-1061<o:p></o:p></pre><pre>Email: <a href="mailto:weng@cse.msu.edu">weng@cse.msu.edu</a><o:p></o:p></pre><pre>URL: <a href="http://www.cse.msu.edu/~weng/">http://www.cse.msu.edu/~weng/</a><o:p></o:p></pre><pre>----------------------------------------------<o:p></o:p></pre><pre><o:p> </o:p></pre></div>
<br /><br />
<hr style='border:none; color:#909090; background-color:#B0B0B0; height: 1px; width: 99%;' />
<table style='border-collapse:collapse;border:none;'>
<tr>
<td style='border:none;padding:0px 15px 0px 8px'>
<a href="http://www.avast.com/">
<img border=0 src="http://static.avast.com/emails/avast-mail-stamp.png" />
</a>
</td>
<td>
<p style='color:#3d4d5a; font-family:"Calibri","Verdana","Arial","Helvetica"; font-size:12pt;'>
Detta epostmeddelande innehåller inget virus eller annan skadlig kod för <a href="http://www.avast.com/">avast! Antivirus</a> är aktivt.
</p>
</td>
</tr>
</table>
<br />
</body></html>