Connectionists: Brain-like computing fanfare and big data fanfare
Hava Siegelmann
hava at cs.umass.edu
Sun Jan 26 16:52:11 EST 2014
nicely said.
On 1/26/14 2:43 PM, Geoffrey Hinton wrote:
> I can no longer resist making one point.
>
> A lot of the discussion is about telling other people what they should
> NOT be doing. I think people should just get on and do whatever they
> think might work. Obviously they will focus on approaches that make
> use of their particular skills. We won't know until afterwards which
> approaches led to major progress and which were dead ends. Maybe a
> fruitful approach is to model every connection in a piece of retina
> in order to distinguish between detailed theories of how cells get to
> be direction selective. Maybe its building huge and very artificial
> neural nets that are much better than other approaches at some
> difficult task. Probably its both of these and many others too. The
> way to really slow down the expected rate of progress in understanding
> how the brain works is to insist that there is one right approach and
> nearly all the money should go to that approach.
>
> Geoff
>
>
>
> On Sat, Jan 25, 2014 at 3:00 PM, Brad Wyble <bwyble at gmail.com
> <mailto:bwyble at gmail.com>> wrote:
>
> I am extremely pleased to see such vibrant discussion here and my
> thanks to Juyang for getting the ball rolling.
>
> Jim, I appreciate your comments and I agree in large measure, but
> I have always disagreed with you as regards the necessity of
> simulating everything down to a lowest common denominator . Like
> you, I enjoy drawing lessons from the history of other
> disciplines, but unlike you, I don't think the analogy between
> neuroscience and physics is all that clear cut. The two fields
> deal with vastly different levels of complexity and therefore I
> don't think it should be expected that they will (or should)
> follow the same trajectory.
>
> To take your Purkinje cell example, I imagine that there are those
> who view any such model that lacks an explicit simulation of the
> RNA as being incomplete. To such a person, your models would also
> be unfit for the literature. So would we then change the standards
> such that no model can be published unless it includes an explicit
> simulation of the RNA? And why stop there? Where does it end?
> In my opinion, we can't make effective progress in this field if
> everyone is bound to the molecular level.
>
> I really think that neuroscience presents a fundamental challenge
> that is not present in physics, which is that progress can only
> occur when theory is developed at different levels of abstraction
> that overlap with one another. The challenge is not how to force
> everyone to operate at the same level of formal specificity, but
> how to allow effective communication between researchers operating
> at different levels.
>
> In aid of meeting this challenge, I think that our field should
> take more inspiration from engineering, a model-based discipline
> that already has to work simultaneously at many different scales
> of complexity and abstraction.
>
>
> Best,
> Brad Wyble
>
>
>
>
> On Sat, Jan 25, 2014 at 9:59 AM, james bower <bower at uthscsa.edu
> <mailto:bower at uthscsa.edu>> wrote:
>
> Thanks for your comments Thomas, and good luck with your effort.
>
> I can’t refrain myself from making the probably culturist
> remark that this seems a very practical approach.
>
> I have for many years suggested that those interested in
> advancing biology in general and neuroscience in particular to
> a ‘paradigmatic’ as distinct from a descriptive / folkloric
> science, would benefit from understanding this transition as
> physics went through it in the 15th and 16th centuries. In
> many ways, I think that is where we are today, although with
> perhaps the decided disadvantage that we have a lot of
> physicists around who, again in my view, don’t really
> understand the origins of their own science. By that, I mean,
> that they don’t understand how much of their current
> scientific structure, for example the relatively clean
> separation between ‘theorists’ and ‘experimentalists’, is
> dependent on the foundation build by those (like Newton) who
> were both in an earlier time. Once you have a sold underlying
> computational foundation for a science, then you have the
> luxury of this kind of specialization - as there is a
> framework that ties it all together. The Higgs effort being a
> very visible recent example.
>
> Neuroscience has nothing of the sort. As I point out in the
> article I linked to in my first posting - while it was first
> proposed 40 years ago (by Rodolfo Llinas) that the cerebellar
> Purkinje cell had active dendrites (i.e. that there were non
> directly-synaptically associated voltage dependent ion
> channels in the dendrite that governed its behavior), and 40
> years of anatomically and physiologically realistic modeling
> has been necessary to start to understand what they do - many
> cerebellar modeling efforts today simply ignore these
> channels. While that again, to many on this list, may seem
> too far buried in the details, these voltage dependent
> channels make the Purkinje cell the computational device that
> it is.
>
> Recently, I was asked to review a cerebellar modeling paper in
> which the authors actually acknowledged that their model
> lacked these channels because they would have been too
> computationally expensive to include. Sadly for those
> authors, I was asked to review the paper for the usual reason
> - that several of our papers were referenced accordingly.
> They likely won’t make that mistake again - as after of
> course complementing them on the fact that they were honest
> (and knowledgable) enough to have remarked on the fact that
> their Purkinje cells weren’t really Purkinje cells - I had to
> reject the paper for the same reason.
>
> As I said, they likely won’t make that mistake again - and
> will very likely get away with it.
>
> Imagine a comparable situation in a field (like physics) which
> has established a structural base for its enterprise. “We
> found it computational expedient to ignore the second law of
> thermodynamics in our computations - sorry”. BTW, I know that
> details are ignored all the time in physics as one deals with
> descriptions at different levels of scale - although even
> there, the field clearly would like to have a way to link
> across different levels of scale. I would claim, however,
> that that is precisely the “trick’ that biology uses to ‘beat’
> the second law - linking all levels of scale together -
> another reason why you can’t ignore the details in biological
> models if you really want to understand how biology works.
> (too cryptic a comment perhaps).
>
> Anyway, my advice would be to consider how physics made this
> transition many years ago, and ask the question how
> neuroscience (and biology) can now. Key points I think are:
> - you need to produce students who are REALLY both
> experimental and theoretical (like Newton). (and that doesn’t
> mean programs that “import” physicists and give them enough
> biology to believe they know what they are doing, or programs
> that link experimentalists to physicists to solve their
> computational problems)
> - you need to base the efforts on models (and therefore
> mathematics) of sufficient complexity to capture the physical
> reality of the system being studied (as Kepler was forced to
> do to make the sun centric model of the solar system even as
> close to as accurate as the previous earth centered system)
> - you need to build a new form of collaboration and
> communication that can support the complexity of those models.
> Fundamentally, we continue to use the publication system
> (short papers in a journal) that was invented as part of the
> transformation for physics way back then. Our laboratories
> are also largely isolated and non-cooperative, more
> appropriate for studying simpler things (like those in
> physics). Fortunate for us, we have a new communication tool
> (the Internet) although, as can be expected, we are mostly
> using it to reimplement old style communication systems
> (e-journals) with a few twists (supplemental materials).
> - funding agencies need to insist that anyone doing theory
> needs to be linked to the experimental side REALLY, and vice
> versa. I proposed a number of years ago to NIH that they
> would make it into the history books if they simply required
> the following monday, that any submitted experimental grant
> include a REAL theoretical and computational component -
> Sadly, they interpreted that as meaning that P.I.s should
> state "an hypothesis" - which itself is remarkable, because
> most of the ‘hypotheses’ I see stated in Federal grants are
> actually statements of what the P.I. believes to be true.
> Don’t get me started on human imaging studies. arggg
> - As long as we are talking about what funding agencies can
> do, how about the following structure for grants - all grants
> need to be submitted collaboratively by two laboratories who
> have different theories (better models) about how a particular
> part of the brain works. The grant should support at set of
> experiments, that both parties agree distinguish between their
> two points of view. All results need to be published with
> joint authorship. In effect that is how physics works - given
> its underlying structure.
> - You need to get rid, as quickly as possible, the pressure to
> “translate” neuroscience research explicitly into clinical
> significance - we are not even close to being able to do that
> intentionally - and the pressure (which is essentially a give
> away to the pharma and bio-tech industries anyway) is forcing
> neurobiologists to link to what is arguably the least
> scientific form of research there is - clinical research. It
> just has to be the case that society needs to understand that
> an investment in basic research will eventually result in all
> the wonderful outcomes for humans we would all like, but this
> distortion now is killing real neuroscience just at a critical
> time, when we may finally have the tools to make the
> transition to a paradigmatic science.
> As some of you know, I have been all about trying to do these
> things for many years - with the GENESIS project, with the
> original CNS graduate program at Caltech, with the CNS
> meetings, (even originally with NIPS) and with the first
> ‘Methods in Computational Neuroscience Course" at the Marine
> Biological laboratory, whose latest incarnation in Brazil
> (LASCON) is actually wrapping up next week, and of course with
> my own research and students. Of course, I have not been
> alone in this, but it is remarkable how little impact all that
> has had on neuroscience or neuro-engineering. I have to say,
> honestly, that the strong tendency seems to be for these
> efforts to snap back to the non-realistic, non-biologically
> based modeling and theoretical efforts.
>
> Perhaps Canada, in its usual practical and reasonable way
> (sorry) can figure out how to do this right.
>
> I hope so.
>
> Jim
>
> p.s. I have also been proposing recently that we scuttle the
> ‘intro neuroscience’ survey courses in our graduate programs
> (religious instruction) and instead organize an introductory
> course built around the history of the discovery of the origin
> of the axon potential that culminated in the first (and last)
> Nobel prize work in computational neuroscience for the Hodkin
> Huxley model. The 50th anniversary of that prize was
> celebrated last year, and the year before I helped to organize
> a meeting celebrating the 60th anniversary of the publication
> of the original papers (which I care much more about anyway).
> That meeting was, I believe, the first meeting in
> neuroscience ever organized around a single (mathematical)
> model or theory - and in organizing it, I required all the
> speakers to show the HH model on their first slide, indicating
> which term or feature of the model their work was related to.
> Again, a first - but possible, as this is about the only
> “community model’ we have.
>
> Most Neuroscience textbooks today don’t include that equation
> (second order differential) and present the HH model primarily
> as a description of the action potential. Most theorists
> regard the HH model as a prime example of how progress can be
> made by ignoring the biological details. Both views and
> interpretations are historically and practically incorrect.
> In my opinion, if you can’t handle the math in the HH model,
> you shouldn’t be a neurobiologist, and if you don’t understand
> the profound impact of HH’s knowledge and experimental study
> of the squid giant axon on the model, you shouldn’t be a
> neuro-theorist either. just saying. :-)
>
>
> On Jan 25, 2014, at 6:58 AM, Thomas Trappenberg <tt at cs.dal.ca
> <mailto:tt at cs.dal.ca>> wrote:
>
>> James, enjoyed your writing.
>>
>> So, what to do? We are trying to get organized in Canada and
>> are thinking how we fit in with your (US) and the European
>> approaches and big money. My thought is that our advantage
>> might be flexibility by not having a single theme but rather
>> a general supporting structure for theory and
>> theory-experimental interactions. I believe the ultimate
>> place where we want to be is to take theoretical proposals
>> more seriously and try to make specific experiments for them;
>> like the Higgs project. (Any other suggestions? Canadians,
>> see http://www.neuroinfocomp.ca
>> <http://www.neuroinfocomp.ca/> if you are not already on there.)
>>
>> Also, with regards to big data, I believe that one very
>> fascinating thing about the brain is that it can function
>> with 'small data'.
>>
>> Cheers, Thomas
>>
>>
>> On 2014-01-25 12:09 AM, "james bower" <bower at uthscsa.edu
>> <mailto:bower at uthscsa.edu>> wrote:
>>
>> Ivan thanks for the response,
>>
>> Actually, the talks at the recent Neuroscience Meeting
>> about the Brain Project either excluded modeling
>> altogether - or declared we in the US could leave it to
>> the Europeans. I am not in the least bit nationalistic -
>> but, collecting data without having models (rather than
>> imaginings) to indicate what to collect, is simply
>> foolish, with many examples from history to demonstrate
>> the foolishness. In fact, one of the primary proponents
>> (and likely beneficiaries) of this Brain Project, who
>> gave the big talk at Neuroscience on the project (showing
>> lots of pretty pictures), started his talk by asking:
>> “what have we really learned since Cajal, except that
>> there are also inhibitory neurons?” Shocking, not only
>> because Cajal actually suggested that there might be
>> inhibitory neurons - in fact. To quote “Stupid is as
>> stupid does”.
>>
>> Forbes magazine estimated that finding the Higgs Boson
>> cost over $13BB, conservatively. The Higgs experiment
>> was absolutely the opposite of a Big Data experiment - In
>> fact, can you imagine the amount of money and time that
>> would have been required if one had simply decided to
>> collect all data at all possible energy levels? The Higgs
>> experiment is all the more remarkable because it had the
>> nearly unified support of the high energy physics
>> community, not that there weren’t and aren’t skeptics,
>> but still, remarkable that the large majority could agree
>> on the undertaking and effort. The reason is, of course,
>> that there was a theory - that dealt with the particulars
>> and the details - not generalities. In contrast, there
>> is a GREAT DEAL of skepticism (me included) about the
>> Brain Project - its politics and its effects (or lack
>> therefore), within neuroscience. (of course, many people
>> are burring their concerns in favor of tin cups -
>> hoping). Neuroscience has had genome envy for ever - the
>> connectome is their response - who says its all in the
>> connections? (sorry ‘connectionists’) Where is the
>> theory? Hebb? You should read Hebb if you haven’t -
>> rather remarkable treatise. But very far from a theory.
>>
>> If you want an honest answer to your question - I have
>> not seen any good evidence so far that the approach
>> works, and I deeply suspect that the nervous system is
>> very much NOT like any machine we have built or designed
>> to date. I don’t believe that Newton would have
>> accomplished what he did, had he not, first, been a
>> remarkable experimentalist, tinkering with real things.
>> I feel the same way about Neuroscience. Having spent
>> almost 30 years building realistic models of its cells
>> and networks (and also doing experiments, as described in
>> the article I linked to) we have made some small progress
>> - but only by avoiding abstractions and paying attention
>> to the details. OF course, most experimentalists and
>> even most modelers have paid little or no attention. We
>> have a sociological and structural problem that, in my
>> opinion, only the right kind of models can fix, coupled
>> with a real commitment to the biology - in all its
>> complexity. And, as the model I linked tries to make
>> clear - we also have to all agree to start working on
>> common “community models’. But like big horn sheep, much
>> safer to stand on your own peak and make a lot of noise.
>>
>> You can predict with great accuracy the movement of the
>> planets in the sky using circles linked to other circles
>> - nice and easy math, and very adaptable model (just add
>> more circles when you need more accuracy, and invent
>> entities like equant points, etc). Problem is, without
>> getting into the nasty math and reality of ellipses- you
>> can’t possible know anything about gravity, or the
>> origins of the solar system, or its various and eventual
>> perturbations.
>>
>> As I have been saying for 30 years: Beware Ptolemy and
>> curve fitting.
>>
>> The details of reality matter.
>>
>> Jim
>>
>>
>>
>>
>>
>> On Jan 24, 2014, at 7:02 PM, Ivan Raikov
>> <ivan.g.raikov at gmail.com
>> <mailto:ivan.g.raikov at gmail.com>> wrote:
>>
>>>
>>> I think perhaps the objection to the Big Data approach
>>> is that it is applied to the exclusion of all other
>>> modelling approaches. While it is true that complete and
>>> detailed understanding of neurophysiology and anatomy is
>>> at the heart of neuroscience, a lot can be learned about
>>> signal propagation in excitable branching structures
>>> using statistical physics, and a lot can be learned
>>> about information representation and transmission in the
>>> brain using mathematical theories about distributed
>>> communicating processes. As these modelling approaches
>>> have been successfully used in various areas of science,
>>> wouldn't you agree that they can also be used to
>>> understand at least some of the fundamental properties
>>> of brain structures and processes?
>>>
>>> -Ivan Raikov
>>>
>>> On Sat, Jan 25, 2014 at 8:31 AM, james bower
>>> <bower at uthscsa.edu <mailto:bower at uthscsa.edu>> wrote:
>>>
>>> [snip]
>>>
>>> An enormous amount of engineering and neuroscience
>>> continues to think that the feedforward pathway is
>>> from the sensors to the inside - rather than seeing
>>> this as the actual feedback loop. Might to some
>>> sound like a semantic quibble, but I assure you it
>>> is not.
>>>
>>> If you believe as I do, that the brain solves very
>>> hard problems, in very sophisticated ways, that
>>> involve, in some sense the construction of complex
>>> models about the world and how it operates in the
>>> world, and that those models are manifest in the
>>> complex architecture of the brain - then simplified
>>> solutions are missing the point.
>>>
>>> What that means inevitably, in my view, is that the
>>> only way we will ever understand what brain-like is,
>>> is to pay tremendous attention experimentally and in
>>> our models to the actual detailed anatomy and
>>> physiology of the brains circuits and cells.
>>>
>>
>> Dr. James M. Bower Ph.D.
>>
>> Professor of Computational Neurobiology
>>
>> Barshop Institute for Longevity and Aging Studies.
>>
>> 15355 Lambda Drive
>>
>> University of Texas Health Science Center
>>
>> San Antonio, Texas 78245
>>
>>
>> *Phone: 210 382 0553 <tel:210%20382%200553>*
>>
>> Email: bower at uthscsa.edu <mailto:bower at uthscsa.edu>
>>
>> Web: http://www.bower-lab.org <http://www.bower-lab.org/>
>>
>> twitter: superid101
>>
>> linkedin: Jim Bower
>>
>>
>> CONFIDENTIAL NOTICE:
>>
>> The contents of this email and any attachments to it may
>> be privileged or contain privileged and confidential
>> information. This information is only for the viewing or
>> use of the intended recipient. If you have received
>> this e-mail in error or are not the intended recipient,
>> you are hereby notified that any disclosure, copying,
>> distribution or use of, or the taking of any action in
>> reliance upon, any of the information contained in this
>> e-mail, or
>>
>> any of the attachments to this e-mail, is strictly
>> prohibited and that this e-mail and all of the
>> attachments to this e-mail, if any, must be
>>
>> immediately returned to the sender or destroyed and, in
>> either case, this e-mail and all attachments to this
>> e-mail must be immediately deleted from your computer
>> without making any copies hereof and any and all hard
>> copies made must be destroyed. If you have received this
>> e-mail in error, please notify the sender by e-mail
>> immediately.
>>
>>
>>
>
> Dr. James M. Bower Ph.D.
>
> Professor of Computational Neurobiology
>
> Barshop Institute for Longevity and Aging Studies.
>
> 15355 Lambda Drive
>
> University of Texas Health Science Center
>
> San Antonio, Texas 78245
>
> *Phone: 210 382 0553 <tel:210%20382%200553>*
>
> Email: bower at uthscsa.edu <mailto:bower at uthscsa.edu>
>
> Web: http://www.bower-lab.org
>
> twitter: superid101
>
> linkedin: Jim Bower
>
> CONFIDENTIAL NOTICE:
>
> The contents of this email and any attachments to it may be
> privileged or contain privileged and confidential information.
> This information is only for the viewing or use of the
> intended recipient. If you have received this e-mail in error
> or are not the intended recipient, you are hereby
> notified that any disclosure, copying, distribution or use of,
> or the taking of any action in reliance upon, any of the
> information contained in this e-mail, or
>
> any of the attachments to this e-mail, is strictly prohibited
> and that this e-mail and all of the attachments to this
> e-mail, if any, must be
>
> immediately returned to the sender or destroyed and, in either
> case, this e-mail and all attachments to this e-mail must be
> immediately deleted from your computer without making any
> copies hereof and any and all hard copies made must be
> destroyed. If you have received this e-mail in error,
> please notify the sender by e-mail immediately.
>
>
>
>
>
> --
> Brad Wyble
> Assistant Professor
> Psychology Department
> Penn State University
>
> http://wyblelab.com
>
>
--
Hava T. Siegelmann, Ph.D.
Professor
Director, BINDS Lab (Biologically Inspired Neural Dynamical Systems)
Dept. of Computer Science
Program of Neuroscience and Behavior
University of Massachusetts Amherst
Amherst, MA, 01003
Phone - Grant Administrator – Michele Roberts: 413-545-4389
Fax: 413-545-1249
LAB WEBSITE: http://binds.cs.umass.edu/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140126/f776d686/attachment.html>
More information about the Connectionists
mailing list