Connectionists: Brain-like computing fanfare and big data fanfare

[james bower]

Hi Paul,

Good to hear from you - as usual measured and rational.  :-)

I have often felt that the better way to think about Neural Network and Connectionist type efforts is more in defining what is likely not Brain-like, rather than claiming they are ‘brain-like”.  I have no problem what-so-ever with these sorts of efforts providing cautions and constraints on thinking about brains - almost in contrast.  I have always had a big problem with the opposite.

Years ago, at one of the first Snow Bird meetings, there was someone whose name I no longer remember, but he was from MIT (remember that) who gave a quite interesting talk.  He had used a combinatorial approach to make a large set of NN models and had predicted that a larger proportion of them (as I remember 95%) would naturally produce oscillations.  Given the problem in engineering with things that oscillate, he suggested that the NN community should be looking to define and work with the small percentage that don’t.  I believe I stood up and said something like:  “nice observation, wrong conclusion”.

Although another soap box (way too many, I think one accumulates them over time), it is quite clear now to many that brains use the oscillatory properties of their networks intrinsically. The soap box is the relative lack of real progress in figuring that out, given the sociological structure of neuroscience, and the love of abstraction.

:-)

Jim



On Jan 25, 2014, at 6:38 AM, Paul Adams <paul.adams at stonybrook.edu> wrote:

> As a former synaptic physiologist who now dabbles in neural net models, I have a slightly different take on this interesting debate. One can argue the new improved neural networks (and more broadly computing and machine learning progress) can achieve brain-like performance without detailed biological imitation or knowledge. One can also argue that direct and intensive study of the brain itself will be necessary, and will reveal major new principles. However, the universal core of connectionism is the idealization that local activity-dependent adjustment of synaptic weights is both crucial and achievable (on a small scale in digital simulations, and on a much larger scale in real brains). However we and also Terry Elliott (see NeCo 24,455-522) have shown that even for the simplest type of unsupervised learning (classic ICA) even negligible deviations from ideality can prevent learning. In other words, the entire connectionist project might be built on shifting sands. But, perhaps more likely and more interestingly, even though it's built on sand, much of the complication one sees in real brains might be the way nature nevertheless builds massive structures on such shifting sand. This would mean that both viewpoints are correct, and the hard problem is to combine the two.
> - Paul Adams
> Department of Neurobiology, Stony Brook University
> 
> 
> 
> On Fri, Jan 24, 2014 at 10:54 PM, james bower <bower at uthscsa.edu> wrote:
> Ivan thanks for the response,
> 
> Actually, the talks at the recent Neuroscience Meeting about the Brain Project either excluded modeling altogether  -  or declared we in the US could leave it to the Europeans.  I am not in the least bit nationalistic - but, collecting data without having models (rather than imaginings) to indicate what to collect, is simply foolish, with many examples from history to demonstrate the foolishness.  In fact, one of the primary proponents (and likely beneficiaries) of this Brain Project, who gave the big talk at Neuroscience on the project (showing lots of pretty pictures), started his talk by asking: “what have we really learned since Cajal, except that there are also inhibitory neurons?”  Shocking, not only because Cajal actually suggested that there might be inhibitory neurons - in fact.  To quote “Stupid is as stupid does”.
> 
> Forbes magazine estimated that finding the Higgs Boson cost over $13BB, conservatively.  The Higgs experiment was absolutely the opposite of a Big Data experiment - In fact, can you imagine the amount of money and time that would have been required if one had simply decided to collect all data at all possible energy levels?   The Higgs experiment is all the more remarkable because it had the nearly unified support of the high energy physics community, not that there weren’t and aren’t skeptics, but still, remarkable that the large majority could agree on the undertaking and effort.  The reason is, of course, that there was a theory - that dealt with the particulars and the details - not generalities.  In contrast, there is a GREAT DEAL of skepticism (me included) about the Brain Project - its politics and its effects (or lack therefore), within neuroscience.  (of course, many people are burring their concerns in favor of tin cups - hoping).  Neuroscience has had genome envy for ever - the connectome is their response - who says its all in the connections? (sorry ‘connectionists’)  Where is the theory?  Hebb?  You should read Hebb if you haven’t - rather remarkable treatise.  But very far from a theory.
> 
> If you want an honest answer to your question - I have not seen any good evidence so far that the approach works, and I deeply suspect that the nervous system is very much NOT like any machine we have built or designed to date. I don’t believe that Newton would have accomplished what he did, had he not, first, been a remarkable experimentalist, tinkering with real things.  I feel the same way about Neuroscience.  Having spent almost 30 years building realistic models of its cells and networks (and also doing experiments, as described in the article I linked to) we have made some small progress - but only by avoiding abstractions and paying attention to the details.  OF course, most experimentalists and even most modelers have paid little or no attention.  We have a sociological and structural problem that, in my opinion, only the right kind of models can fix, coupled with a real commitment to the biology - in all its complexity.  And, as the model I linked tries to make clear - we also have to all agree to start working on common “community models’.  But like big horn sheep, much safer to stand on your own peak and make a lot of noise.  
> 
> You can predict with great accuracy the movement of the planets in the sky using circles linked to other circles - nice and easy math, and very adaptable model (just add more circles when you need more accuracy, and invent entities like equant points, etc).  Problem is, without getting into the nasty math and reality of ellipses- you can’t possible know anything about gravity, or the origins of the solar system, or its various and eventual perturbations.  
> 
> As I have been saying for 30 years:  Beware Ptolemy and curve fitting.
> 
> The details of reality matter.
> 
> Jim
> 
> 
> 
> 
> 
> On Jan 24, 2014, at 7:02 PM, Ivan Raikov <ivan.g.raikov at gmail.com> wrote:
> 
>> 
>> I think perhaps the objection to the Big Data approach is that it is applied to the exclusion of all other modelling approaches. While it is true that complete and detailed understanding of  neurophysiology and anatomy is at the heart of neuroscience, a lot can be learned about signal propagation in excitable branching structures using statistical physics, and a lot can be learned about information representation and transmission in the brain using mathematical theories about distributed communicating processes. As these modelling approaches have been successfully used in various areas of science, wouldn't you agree that they can also be used to understand at least some of the fundamental properties of brain structures and processes? 
>> 
>>   -Ivan Raikov
>> 
>> On Sat, Jan 25, 2014 at 8:31 AM, james bower <bower at uthscsa.edu> wrote:
>> [snip] 
>> An enormous amount of engineering and neuroscience continues to think that the feedforward pathway is from the sensors to the inside - rather than seeing this as the actual feedback loop.  Might to some sound like a semantic quibble,  but I assure you it is not.
>> 
>> If you believe as I do, that the brain solves very hard problems, in very sophisticated ways, that involve, in some sense the construction of complex models about the world and how it operates in the world, and that those models are manifest in the complex architecture of the brain - then simplified solutions are missing the point.
>> 
>> What that means inevitably, in my view, is that the only way we will ever understand what brain-like is, is to pay tremendous attention experimentally and in our models to the actual detailed anatomy and physiology of the brains circuits and cells.
>> 
> 
>  
> 
>  
> 
> Dr. James M. Bower Ph.D.
> 
> Professor of Computational Neurobiology
> 
> Barshop Institute for Longevity and Aging Studies.
> 
> 15355 Lambda Drive
> 
> University of Texas Health Science Center 
> 
> San Antonio, Texas  78245
> 
>  
> Phone:  210 382 0553
> 
> Email: bower at uthscsa.edu
> 
> Web: http://www.bower-lab.org
> 
> twitter: superid101
> 
> linkedin: Jim Bower
> 
>  
> CONFIDENTIAL NOTICE:
> 
> The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or
> 
> any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be
> 
> immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.
> 
>  
> 
> 
> 
> 
> -- 
> Paul Adams
> Department of Neurobiology
> Stony Brook University

 

 

Dr. James M. Bower Ph.D.

Professor of Computational Neurobiology

Barshop Institute for Longevity and Aging Studies.

15355 Lambda Drive

University of Texas Health Science Center 

San Antonio, Texas  78245

 

Phone:  210 382 0553

Email: bower at uthscsa.edu

Web: http://www.bower-lab.org

twitter: superid101

linkedin: Jim Bower

 

CONFIDENTIAL NOTICE:

The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or

any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be

immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140125/bf713c7f/attachment.html>