Connectionists: Physics and Psychology (and the C-word)

[james bower]

On Jan 28, 2014, at 3:34 PM, Carson Chow <ccchow at pitt.edu> wrote:

> Jim,
> 
> Before you check out there is one question that I wanted you to address. I think you believe that the brain is irreducibly complex so that Kolmogorov complexity of the brain is the brain itself.  Is this true? 

Yes, this is my operating assumption.  One thing to note about that, that I think is important.  In my view a major issue with neuroscience is that I believe we actually have a very poor understanding of what the brain actually does.  While we have technology that can study its anatomical structure at angstrom spacial scale, and we routinely measure its electrical activity at microsecond and millisecond levels, in most cases our description of its overall behavior, is at scales of minutes to hours and not very sophisticated at all.  Yes, we can measure physical movement of eyes, limbs, etc at very high spatial and temporal resolution - and clearly progress has been made in linking those kinds of brain behavior to neural activity (although perhaps not as much as some think) -  however at the level of the behavioral function of the overall machine I believe our understanding is far too crude and as challenging as figuring out the brains structure (another way to answer your question). (Branch point I won’t take  to how much ‘cognitive’ analysis is responsible for this :-) ). 

Anyway, How is that related to the complexity question?  One can certainly design a model or system that, for example, controls the movements of robotic limbs that has much less the apparent complexity of the systems in the brain that control arm movement.  While I think it is still fair to say that after many years of effort, robots designed to mimic the behavior of actual biological organisms are still clearly distinguishable from the biological case.  However, it seems to me that figuring out IF the Komogorov complexity of the problem the brain solves, is equivalent to the complexity of the brain, requires an understanding of the complexity of the problem the brain solves in total. Of course, in my animal behavior course, students eventually figure out that in some overall sense, the problem that male and female brains are trying to solve is whether he/she will or won’t let me commingle gametes (why it takes 19 year olds half a semester to realize that has always been beyond me).  However, in fact, the behavior, decision making etc involved in getting to that point and then beyond is vastly more complex I think, than we know. (perhaps one reason why we like to short circuit all that stuff with drugs and rock and roll  :-)  ).

But why do I assume the K-C relationship:

As I mentioned before, in those few cases were we can actually authoritatively measure the performance of the nervous system against constraints given by the actual physics (mostly those measurements have been made at the receptor level), the nervous system’s performance comes right up against the physical limits.  NOT ONLY THAT, there is evidence that further computation in the brain can actually improve the resolution (so-called hyper acuity).

There is also a thermodynamic argument - very simple (perhaps too simple).  The brain has 10 to the 12th neurons (lets say), 10 to the 14th (conservatively) connections.  And its abilities are obviously remarkable.  Yet you have to heat it up to make it work.  It doesn’t generate enough energy to keep itself warm.  Perhaps everyone doesn’t know that, by volume> the largest fraction of the brain (by a lot) is actually the system that supports the neurons - the glia and the vascular system.  Providing the energy (Glucose remarkably enough) and the heat.  

Sounds to me like rather remarkable engineering (if  you excuse the inference).

As everyone here probably does know, heat generation is a principle limitation on the design of modern chips - 

One other overall measure of this structures likely sophistication.  The first talk I gave in a Neural Networks context, was at a small meeting at the Miramar Hotel in Santa Barbara.  This meeting had been organized (by Caltech and AT&T) as a follow on to a ‘Hopfest” that had taken place a few months earlier at Caltech.  I gave a talk about the structure of olfactory cortex and the likely origin of the theta (7-12 Hz) and Gama (40Hz) oscillations in cerebral cortex - by talking about the results of what i think was the first anatomically realistic model of the cerebral cortex that had been built (by Matt Wilson as a master’s degree student on an IBM XT computer).  In describing the model, I stated that there appeared to be a very diffuse pattern of intrinsic connections with no order (note my previous post, when I said that I now believe based on the 3rd generation of that model that this wrong - anyway).  The audience didn’t care about oscillations at all - but they did pick  up on the idea that there was possibly a Hopflied-Like network in the real brain. and they asked if this was possible.  I told them that it was almost certainly not possible, but that evening in the bar “proved it” by calculating how large the brain would be if all the neurons were connected to every other neuron.  Only considering the axons - excluding the rest of the neurons, the glia, the vasculature etc, the answer turned out to be 20 KM in diameter.  

Not much room for redundancy there. 

So, sorry, once again long winded - but my assumption for all these reasons is that the K-complexity of the brain, is the brain itself.  (and let the howling begin).


> If this is true then the only model of the brain is the brain itself.  We can reconstruct it by faithfully simulating it but there is no abstraction of it that is simpler than the whole thing, right? In such a situation, what does it mean to understand the brain? 

An outstanding and important question and something I worry about all the time while, at the same time, I don’t.  And here is why.  And this, I think is a critical distinction in the world of modeling AND in this debate.  It is not my intent to publish in my lifetime an accurate model of the brain and its function, or even the cerebellar Purkinje cell and I don’t think anyone else will either.  My interest has always been in the process - in being on the right path - that is really at the fundamental foundation of what I have been trying to say for the last 3 days.  We need to think of this endeavor as a quest - and I don’t think there will be any short cuts.  Building GENESIS, working to start the CNS graduate program at Caltech, starting NIPS, starting the CNS program, starting the summer courses in Woods Hole and then Europe and then latin America, starting the Journal of Computational Neuroscience, publishing all our models openly, inviting criticism, etc - was all about starting a process - which I think we are at the very beginning of - absolutely.  It is my deep belief that if we as humans start on a quest, with the right tools, headed in the right direction, working collaborative and cooperatively, we can make progress.  We did build the pyramids, thousands of years ago.  Pretty remarkable.  But I think throughout human history, we have often tried to take short cuts - and we also have a strong tendency to let our egos drive the enterprise - to be the one who found out how visual cortex worked, or at least convinced some number of poor experimentalists that we are right.  In my view, not much chance.  So, in fact, those on this list who have criticized me for claiming that there is one true way (my way) to figuring this system out, are not understanding.  All I said, and what I deeply believe, is if your model or theory isn’t directly connected to the actual physical structure of the machine - it isn’t likely to contribute to the long haul process of figuring out how that machine works.  And if your experimental efforts, aren’t linked to models and theories, you have no way of knowing that the data you are collecting are actually the information that is most needed now to advance our understanding for the next increment. And if the strong tendency of the field is to have everyone work on their own models, claim their models account for all the data (just like everyone else tend to claim their models do) and at the same time claim their model is unique - then - we aren’t using the right process.  And I think that progress is all about process.  

Anyway, I am being redundant - but I think that your question is fundamental, and also honestly why there is so much resistance to this approach to modeling - in our heart of hearts I suspect we all know that this task is not doable in our lifetime, and who knows, may not be doable at all.  (Physicist looking for their keys under the light post).   But what the heck - its interesting.


> 
> Also, while I agree that we should focus only on things that evolution can affect I don't think we can say how optimal the brain is. 

see above - and of course we can’t - all we can say is that we haven’t come close to building a machine that does what it does IN TOTAL (yes, we can beat humans at Jeoperdy - but who cares — Watson couldn’t find its way to the studio and certainly couldn’t beat me in polo). Furthermore, I don’t believe that we yet even know what kind of machine the brain is.  I wrote a paper years ago, I think for Trends in Neuroscience, making the point that humans have always thought that the brain was built and functioned based on the same principles as the most sophisticated technology of the day (humors for the aqueduct loving greeks, machinery for DesCarte, and parallel distributed analog computer for us)  are we any closer?  Who knows?  Sadly, we often seem to forget that these comparisons are metaphorical, not actual.



> Three billion years may be a long time to us but nature can only search a miniscule sample of genome space in that time. 

Oh Boy - you really don’t want me to write pages more about how misunderstood the evolutionary process is and how completely unclear it is (in large part because they suffer from the same kind of short cut modeling) how DNA structure is related to cellular behavior - even not considering evolution - remember ‘junk DNA’ - we really don’t want to discuss that.  Another manifestation of the same unfortunate human tendency to take short cuts - (perhaps thist is our tendency is because the brain is always looking for ways to reduce its own energy costs.  :-)  ).

> All we can really say is that the brain is locally optimal conditioned on its entire history.  Thus, we have no idea of how many possible ways to construct brains that have performance capabilities similar to mammals.

one thing we do know - which is another little bit of circumstantial evidence, is that when brain components (like the retina) evolve to perform a similar type of function in two animals without a common seeing ancestor (the Cephlapod and the Primate), the retina that is generated is remarkably similar in structure - in fact, my understanding is that, accept that the Cepholapod retina is pointed in the “right” direction - i.e. towards the light source, it is essentially indistinguishable in its architecture.

Tends to suggest that there may not be that many optimal solutions - at least, it suggests that the same solution evolved  independently. (by the way, the problem of convergent evolution is actually a huge confound, often ignored in computational molding of DNA evolution and history).

Well, I tried to sign off briefly - but couldn’t help myself as I think these are core questions.

The senior faculty on this list have no doubt tired of these kinds of arguments - and many in particular have tired of me making them.  However, I don’t engage in these debates to impress my colleagues (long since unimpressed).  I engage in them so that students can think in new and fresh ways about this stuff, and not bury them, just to get to work.

the question you asked is key - its implications are profound and nobody knows - but, I have my suspicions. 


Best and my apologies for once again, being long winded

Jim 


> 
> best,
> Carson
> 
> 
> 
> On 1/28/14 4:14 PM, james bower wrote:
>> Ok, had enough here - back to work.
>> 
>> It is emblematic, however, for me of the larger problem that a discussion that started out by raising concerns about abstract models, disconnected from the physical realty of machine we are supposed to be understanding, has turned into a debate about quantum theory and consciousness.
>> 
>> I rest my case
>> 
>> The very best to everyone and to all of us as we try to figure this out.  I have no doubt that everyone is sincere and truly believes in the approach they are taking.  For my part, I will stick with the nuts and bolts.
>> 
>> Jim Bower
>> 
>> p.s Last one - personally I take Darwin’s view that the question of consciousness isn’t that interesting.  
>> 
>> 
>> 
>> 
>> On Jan 28, 2014, at 2:50 PM, Richard Loosemore <rloosemore at susaro.com> wrote:
>> 
>>> On 1/28/14, 3:09 PM, Brian J Mingus wrote:
>>>> 
>>>> Hi Richard, thanks for the feedback. 
>>>> 
>>>> > Yes, in general, having an outcome measure that correlates with C ... that is good, but only with a clear and unambigous meaning for C itself (which I don't think anyone has, so therefore it is, after all, of no value to look for outcome measures that correlate)
>>>> 
>>>> Actually, the outcome measure I described is independent of a clear and unambiguous meaning for C itself, and in an interesting way: the models, like us, essentially reinvent the entire literature, and have a conversation as we do, inventing almost all the same positions that we've invented (including the one in your paper). 
>>>> 
>>> 
>>> I can tell you in advance that the theory I propose in that paper makes a prediction there.  If your models (I assume you mean models of the human cognitive system) have precisely the right positioning for their 'concept analysis mechanism' (and they almost certainly would have to... it is difficult to avoid), then they would indeed "reinvent the entire literature, and have a conversation as we do, inventing almost all the same positions that we've invented".
>>> 
>>> However, I can say *why* they should do this, as a tightly-argued consequence of the theory itself, and I can also say why they should express those same confusions about consciousness that we do.  
>>> 
>>> I think that is the key.  I don't think the naked fact that a model-of-cognition reinvents the philosophy of mind would actually tell us anything, sadly.   There is no strong logical compulsion there.  It would boot me little to know that they had done that.
>>> 
>>> Anyhow, look forward to hearing your thoughts if/when you get a chance.
>>> 
>>> Richard Loosemore
>> 
>>  
>> 
>>  
>> 
>> Dr. James M. Bower Ph.D.
>> 
>> Professor of Computational Neurobiology
>> 
>> Barshop Institute for Longevity and Aging Studies.
>> 
>> 15355 Lambda Drive
>> 
>> University of Texas Health Science Center 
>> 
>> San Antonio, Texas  78245
>> 
>>  
>> Phone:  210 382 0553
>> 
>> Email: bower at uthscsa.edu
>> 
>> Web: http://www.bower-lab.org
>> 
>> twitter: superid101
>> 
>> linkedin: Jim Bower
>> 
>>  
>> CONFIDENTIAL NOTICE:
>> 
>> The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or
>> 
>> any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be
>> 
>> immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.
>> 
>>  
>> 
> 

 

 

Dr. James M. Bower Ph.D.

Professor of Computational Neurobiology

Barshop Institute for Longevity and Aging Studies.

15355 Lambda Drive

University of Texas Health Science Center 

San Antonio, Texas  78245

 

Phone:  210 382 0553

Email: bower at uthscsa.edu

Web: http://www.bower-lab.org

twitter: superid101

linkedin: Jim Bower

 

CONFIDENTIAL NOTICE:

The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or

any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be

immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140128/f66cf69d/attachment.html>