Connectionists: Brain-like computing fanfare and big data fanfare

[Brad Wyble]

I am extremely pleased to see such vibrant discussion here and my thanks to
Juyang for getting the ball rolling.

Jim, I appreciate  your comments and I agree in large measure, but I have
always disagreed with you as regards the necessity of simulating everything
down to a lowest common denominator .  Like you, I enjoy drawing lessons
from the history of other disciplines, but unlike you, I don't think the
analogy between neuroscience and physics is all that clear cut.  The two
fields deal with vastly different levels of complexity and therefore I
don't think it should be expected that they will (or should) follow the
same trajectory.

To take your Purkinje cell example, I imagine that there are those who view
any such model that lacks an explicit simulation of the RNA as being
incomplete.  To such a person, your models would also be unfit for the
literature. So would we then change the standards such that no model can be
published unless it includes an explicit simulation of the RNA?  And why
stop there?  Where does it end?  In my opinion, we can't make effective
progress in this field if everyone is bound to the molecular level.

I really think that neuroscience presents a fundamental challenge that is
not present in physics, which is that progress can only occur when theory
is developed at different levels of abstraction that overlap with one
another.  The challenge is not how to force everyone to operate at the same
level of formal specificity, but how to allow effective communication
between researchers operating at different levels.

In aid of meeting this challenge, I think that our field should take more
inspiration from engineering, a  model-based discipline that already has to
work simultaneously at many different scales of complexity and abstraction.


Best,
Brad Wyble




On Sat, Jan 25, 2014 at 9:59 AM, james bower <bower at uthscsa.edu> wrote:

> Thanks for your comments Thomas, and good luck with your effort.
>
> I can’t refrain myself from making the probably culturist remark that this
> seems a very practical approach.
>
> I have for many years suggested that those interested in advancing biology
> in general and neuroscience in particular to a ‘paradigmatic’ as distinct
> from a descriptive / folkloric science, would benefit from understanding
> this transition as physics went through it in the 15th and 16th centuries.
>  In many ways, I think that is where we are today, although with perhaps
> the decided disadvantage that we have a lot of physicists around who, again
> in my view, don’t really understand the origins of their own science.  By
> that, I mean, that they don’t understand how much of their current
> scientific structure, for example the relatively clean separation between
> ‘theorists’ and ‘experimentalists’, is dependent on the foundation build by
> those (like Newton) who were both in an earlier time.  Once you have a sold
> underlying computational foundation for a science, then you have the luxury
> of this kind of specialization - as there is a framework that ties it all
> together.  The Higgs effort being a very visible recent example.
>
> Neuroscience has nothing of the sort.  As I point out in the article I
> linked to in my first posting - while it was first proposed 40 years ago
> (by Rodolfo Llinas) that the cerebellar Purkinje cell had active dendrites
> (i.e. that there were non directly-synaptically associated voltage
> dependent ion channels in the dendrite that governed its behavior), and 40
> years of anatomically and physiologically realistic modeling has been
> necessary to start to understand what they do - many cerebellar modeling
> efforts today simply ignore these channels.  While that again, to many on
> this list, may seem too far buried in the details, these voltage dependent
> channels make the Purkinje cell the computational device that it is.
>
> Recently, I was asked to review a cerebellar modeling paper in which the
> authors actually acknowledged that their model lacked these channels
> because they would  have been too computationally expensive to include.
>  Sadly for those authors, I was asked to review the paper for the usual
> reason - that several of our papers were referenced accordingly.  They
> likely won’t make that mistake again - as after of course complementing
> them on the fact that they were honest (and knowledgable) enough to have
> remarked on the fact that their Purkinje cells weren’t really Purkinje
> cells - I had to reject the paper for the same reason.
>
> As I said, they likely won’t make that mistake again - and will very
> likely get away with it.
>
> Imagine a comparable situation in a field (like physics) which has
> established a structural base for its enterprise.  “We found it
> computational expedient to ignore the second law of thermodynamics in our
> computations - sorry”.  BTW, I know that details are ignored all the time
> in physics as one deals with descriptions at different levels of scale -
> although even there, the field clearly would like to have a way to link
> across different levels of scale.   I would claim, however, that that is
> precisely the “trick’ that biology uses to ‘beat’ the second law - linking
> all levels of scale together - another reason why you can’t ignore the
> details in biological models if  you really want to understand how biology
> works.  (too cryptic a comment perhaps).
>
> Anyway, my advice would be to consider how physics made this transition
> many years ago, and ask the question how neuroscience (and biology) can
> now.  Key points I think are:
> - you need to produce students who are REALLY both experimental and
> theoretical (like Newton).  (and that doesn’t mean programs that “import”
> physicists and give them enough biology to believe they know what they are
> doing, or programs that link experimentalists to physicists to solve their
> computational problems)
> - you need to base the efforts on models (and therefore mathematics) of
> sufficient complexity to capture the physical reality of the system being
> studied (as Kepler was forced to do to make the sun centric model of the
> solar system even as close to as accurate as the previous earth centered
> system)
> - you need to build a new form of collaboration and communication that can
> support the complexity of those models.  Fundamentally, we continue to use
> the publication system (short papers in a journal) that was invented as
> part of the transformation for physics way back then.  Our laboratories are
> also largely isolated and non-cooperative, more appropriate for studying
> simpler things (like those in physics).  Fortunate for us, we have a new
> communication tool (the Internet) although, as can be expected, we are
> mostly using it to reimplement old style communication systems (e-journals)
> with a few twists (supplemental materials).
> - funding agencies need to insist that anyone doing theory needs to be
> linked to the experimental side REALLY, and vice versa.  I proposed a
> number of years ago to NIH that they would make it into the history books
> if they simply required the following monday,  that any submitted
> experimental grant include a REAL theoretical and computational component -
> Sadly, they interpreted that as meaning that P.I.s should state "an
> hypothesis" - which itself is remarkable, because most of the ‘hypotheses’
> I see stated in Federal grants are actually statements of what the P.I.
> believes to be true.  Don’t get me started on human imaging studies.  arggg
> - As long as we are talking about what funding agencies can do, how about
> the following structure for grants - all grants need to be submitted
> collaboratively by two laboratories who have different theories (better
> models) about how a particular part of the brain works.  The grant should
> support at set of experiments, that both parties agree distinguish between
> their two points of view.  All results need to be published with joint
> authorship.  In effect that is how physics works - given its underlying
> structure.
> - You need to get rid, as quickly as possible, the pressure to “translate”
> neuroscience research explicitly into clinical significance - we are not
> even close to being able to do that intentionally - and the pressure (which
> is essentially a give away to the pharma and bio-tech industries anyway) is
> forcing neurobiologists to link to what is arguably the least scientific
> form of research there is - clinical research.  It just has to be the case
> that society needs to understand that an investment in basic research will
> eventually result in all the wonderful outcomes for humans we would all
> like, but this distortion now is killing real neuroscience just at a
> critical time, when we may finally have the tools to make the transition to
> a paradigmatic science.
> As some of you know, I have been all about trying to do these things for
> many years - with the GENESIS project, with the original CNS graduate
> program at Caltech, with the CNS meetings, (even originally with NIPS) and
> with the first  ‘Methods in Computational Neuroscience Course" at the
> Marine Biological laboratory, whose latest incarnation in Brazil (LASCON)
> is actually wrapping up next week, and of course with my own research and
> students.  Of course, I have not been alone in this, but it is remarkable
> how little impact all that has had on neuroscience or neuro-engineering.  I
> have to say, honestly, that the strong tendency seems to be for these
> efforts to snap back to the non-realistic, non-biologically based modeling
> and theoretical efforts.
>
> Perhaps Canada, in its usual practical and reasonable way (sorry) can
> figure out how to do this right.
>
> I hope so.
>
> Jim
>
> p.s. I have also been proposing recently that we scuttle the ‘intro
> neuroscience’ survey courses in our graduate programs (religious
> instruction)  and instead organize an introductory course built around the
> history of the discovery of the origin of the axon potential that
> culminated in the first (and last) Nobel prize work in computational
> neuroscience for the Hodkin Huxley model.  The 50th anniversary of that
> prize was celebrated last year, and the year before I helped to organize a
> meeting celebrating the 60th anniversary of the publication of the original
> papers (which I care much more about anyway).  That meeting was, I believe,
> the first meeting in neuroscience ever organized around a single
> (mathematical) model or theory - and in organizing it, I required all the
> speakers to show the HH model on their first slide, indicating which term
> or feature of the model their work was related to.  Again, a first - but
> possible, as this is about the only “community model’ we have.
>
> Most Neuroscience textbooks today don’t include that equation (second
> order differential) and present the HH model primarily as a description of
> the action potential.   Most theorists regard the HH model as a prime
> example of how progress can be made by ignoring the biological details.
>  Both views and interpretations are historically and practically incorrect.
>  In my opinion, if you can’t handle the math in the HH model, you shouldn’t
> be a neurobiologist, and if you don’t understand the profound impact of
> HH’s knowledge and experimental study of the squid giant axon on the model,
>  you shouldn’t be a neuro-theorist either.  just saying.   :-)
>
>
> On Jan 25, 2014, at 6:58 AM, Thomas Trappenberg <tt at cs.dal.ca> wrote:
>
> James, enjoyed your writing.
>
> So, what to do? We are trying to get organized in Canada and are thinking
> how we fit in with your (US) and the European approaches and big money. My
> thought is that our advantage might be flexibility by not having a single
> theme but rather a general supporting structure for theory and
> theory-experimental interactions. I believe the ultimate place where we
> want to be is to take theoretical proposals more seriously and try to make
> specific experiments for them; like the Higgs project. (Any other
> suggestions? Canadians, see http://www.neuroinfocomp.ca  if you are not
> already on there.)
>
> Also, with regards to big data, I believe that one very fascinating thing
> about the brain is that it can function with 'small data'.
>
> Cheers, Thomas
>
>
> On 2014-01-25 12:09 AM, "james bower" <bower at uthscsa.edu> wrote:
>
>> Ivan thanks for the response,
>>
>> Actually, the talks at the recent Neuroscience Meeting about the Brain
>> Project either excluded modeling altogether  -  or declared we in the US
>> could leave it to the Europeans.  I am not in the least bit nationalistic -
>> but, collecting data without having models (rather than imaginings) to
>> indicate what to collect, is simply foolish, with many examples from
>> history to demonstrate the foolishness.  In fact, one of the primary
>> proponents (and likely beneficiaries) of this Brain Project, who gave the
>> big talk at Neuroscience on the project (showing lots of pretty pictures),
>> started his talk by asking: “what have we really learned since Cajal,
>> except that there are also inhibitory neurons?”  Shocking, not only because
>> Cajal actually suggested that there might be inhibitory neurons - in fact.
>>  To quote “Stupid is as stupid does”.
>>
>> Forbes magazine estimated that finding the Higgs Boson cost over $13BB,
>> conservatively.  The Higgs experiment was absolutely the opposite of a Big
>> Data experiment - In fact, can you imagine the amount of money and time
>> that would have been required if one had simply decided to collect all data
>> at all possible energy levels?   The Higgs experiment is all the more
>> remarkable because it had the nearly unified support of the high energy
>> physics community, not that there weren’t and aren’t skeptics, but still,
>> remarkable that the large majority could agree on the undertaking and
>> effort.  The reason is, of course, that there was a theory - that dealt
>> with the particulars and the details - not generalities.  In contrast,
>> there is a GREAT DEAL of skepticism (me included) about the Brain Project -
>> its politics and its effects (or lack therefore), within neuroscience.  (of
>> course, many people are burring their concerns in favor of tin cups -
>> hoping).  Neuroscience has had genome envy for ever - the connectome is
>> their response - who says its all in the connections? (sorry
>> ‘connectionists’)  Where is the theory?  Hebb?  You should read Hebb if you
>> haven’t - rather remarkable treatise.  But very far from a theory.
>>
>> If you want an honest answer to your question - I have not seen any good
>> evidence so far that the approach works, and I deeply suspect that the
>> nervous system is very much NOT like any machine we have built or designed
>> to date. I don’t believe that Newton would have accomplished what he did,
>> had he not, first, been a remarkable experimentalist, tinkering with real
>> things.  I feel the same way about Neuroscience.  Having spent almost 30
>> years building realistic models of its cells and networks (and also doing
>> experiments, as described in the article I linked to) we have made some
>> small progress - but only by avoiding abstractions and paying attention to
>> the details.  OF course, most experimentalists and even most modelers have
>> paid little or no attention.  We have a sociological and structural problem
>> that, in my opinion, only the right kind of models can fix, coupled with a
>> real commitment to the biology - in all its complexity.  And, as the model
>> I linked tries to make clear - we also have to all agree to start working
>> on common “community models’.  But like big horn sheep, much safer to stand
>> on your own peak and make a lot of noise.
>>
>> You can predict with great accuracy the movement of the planets in the
>> sky using circles linked to other circles - nice and easy math, and very
>> adaptable model (just add more circles when you need more accuracy, and
>> invent entities like equant points, etc).  Problem is, without getting into
>> the nasty math and reality of ellipses- you can’t possible know anything
>> about gravity, or the origins of the solar system, or its various and
>> eventual perturbations.
>>
>> As I have been saying for 30 years:  Beware Ptolemy and curve fitting.
>>
>> The details of reality matter.
>>
>> Jim
>>
>>
>>
>>
>>
>> On Jan 24, 2014, at 7:02 PM, Ivan Raikov <ivan.g.raikov at gmail.com> wrote:
>>
>>
>> I think perhaps the objection to the Big Data approach is that it is
>> applied to the exclusion of all other modelling approaches. While it is
>> true that complete and detailed understanding of  neurophysiology and
>> anatomy is at the heart of neuroscience, a lot can be learned about signal
>> propagation in excitable branching structures using statistical physics,
>> and a lot can be learned about information representation and transmission
>> in the brain using mathematical theories about distributed communicating
>> processes. As these modelling approaches have been successfully used in
>> various areas of science, wouldn't you agree that they can also be used to
>> understand at least some of the fundamental properties of brain structures
>> and processes?
>>
>>   -Ivan Raikov
>>
>> On Sat, Jan 25, 2014 at 8:31 AM, james bower <bower at uthscsa.edu> wrote:
>>
>>> [snip]
>>>
>> An enormous amount of engineering and neuroscience continues to think
>>> that the feedforward pathway is from the sensors to the inside - rather
>>> than seeing this as the actual feedback loop.  Might to some sound like a
>>> semantic quibble,  but I assure you it is not.
>>>
>>> If you believe as I do, that the brain solves very hard problems, in
>>> very sophisticated ways, that involve, in some sense the construction of
>>> complex models about the world and how it operates in the world, and that
>>> those models are manifest in the complex architecture of the brain - then
>>> simplified solutions are missing the point.
>>>
>>> What that means inevitably, in my view, is that the only way we will
>>> ever understand what brain-like is, is to pay tremendous attention
>>> experimentally and in our models to the actual detailed anatomy and
>>> physiology of the brains circuits and cells.
>>>
>>>
>>
>>
>>
>>
>> Dr. James M. Bower Ph.D.
>>
>> Professor of Computational Neurobiology
>>
>> Barshop Institute for Longevity and Aging Studies.
>>
>> 15355 Lambda Drive
>>
>> University of Texas Health Science Center
>>
>> San Antonio, Texas  78245
>>
>>
>> *Phone:  210 382 0553 <210%20382%200553>*
>>
>> Email: bower at uthscsa.edu
>>
>> Web: http://www.bower-lab.org
>>
>> twitter: superid101
>>
>> linkedin: Jim Bower
>>
>>
>> CONFIDENTIAL NOTICE:
>>
>> The contents of this email and any attachments to it may be privileged
>> or contain privileged and confidential information. This information is
>> only for the viewing or use of the intended recipient. If you have received
>> this e-mail in error or are not the intended recipient, you are hereby
>> notified that any disclosure, copying, distribution or use of, or the
>> taking of any action in reliance upon, any of the information contained in
>> this e-mail, or
>>
>> any of the attachments to this e-mail, is strictly prohibited and that
>> this e-mail and all of the attachments to this e-mail, if any, must be
>>
>> immediately returned to the sender or destroyed and, in either case,
>> this e-mail and all attachments to this e-mail must be immediately deleted
>> from your computer without making any copies hereof and any and all hard
>> copies made must be destroyed. If you have received this e-mail in error,
>> please notify the sender by e-mail immediately.
>>
>>
>>
>
>
>
>
> Dr. James M. Bower Ph.D.
>
> Professor of Computational Neurobiology
>
> Barshop Institute for Longevity and Aging Studies.
>
> 15355 Lambda Drive
>
> University of Texas Health Science Center
>
> San Antonio, Texas  78245
>
>
>
> *Phone:  210 382 0553 <210%20382%200553>*
>
> Email: bower at uthscsa.edu
>
> Web: http://www.bower-lab.org
>
> twitter: superid101
>
> linkedin: Jim Bower
>
>
>
> CONFIDENTIAL NOTICE:
>
> The contents of this email and any attachments to it may be privileged
> or contain privileged and confidential information. This information is
> only for the viewing or use of the intended recipient. If you have received
> this e-mail in error or are not the intended recipient, you are hereby
> notified that any disclosure, copying, distribution or use of, or the
> taking of any action in reliance upon, any of the information contained in
> this e-mail, or
>
> any of the attachments to this e-mail, is strictly prohibited and that
> this e-mail and all of the attachments to this e-mail, if any, must be
>
> immediately returned to the sender or destroyed and, in either case,
> this e-mail and all attachments to this e-mail must be immediately deleted
> from your computer without making any copies hereof and any and all hard
> copies made must be destroyed. If you have received this e-mail in error,
> please notify the sender by e-mail immediately.
>
>
>
>


-- 
Brad Wyble
Assistant Professor
Psychology Department
Penn State University

http://wyblelab.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140125/51f5d6d7/attachment.html>