Connectionists: Brain-like computing fanfare and big data fanfare
Stephen José Hanson
jose at psychology.rutgers.edu
Sat Jan 25 10:43:31 EST 2014
Indeed. Its like we never stopped arguing about this for the last 30
years! Maybe this is a brain principle
integrated fossilized views of the brain principles.
I actually agree with John.. and disagree with you JIm... surprise
surprise...seems like old times..
The most disconcerting thing about the emergence the new new neural
network field(s)
is that the NIH Connectome RFPs contain language about large scale
network functions...and
yet when Program managers are directly asked whether fMRI or any
neuroimaging methods
would be compliant with the RFP.. the answer is "NO".
So once the neuroscience cell spikers get done analyzing 1000 or 10000
or even a 1M neurons
at a circuit level.. we still won't know why someone makes decisions
about the shoes they wear; much
less any other mental function! Hopefully neuroimaging will be
relevant again.
Just saying.
Cheers.
Steve
PS. Hi Gary! Dijon!
Stephen José Hanson
Director RUBIC (Rutgers Brain Imaging Center)
Professor of Psychology
Member of Cognitive Science Center (NB)
Member EE Graduate Program (NB)
Member CS Graduate Program (NB)
Rutgers University
email: jose at psychology.rutgers.edu
web: psychology.rutgers.edu/~jose
lab: www.rumba.rutgers.edu
fax: 866-434-7959
voice: 973-353-3313 (RUBIC)
On Fri, 2014-01-24 at 17:31 -0600, james bower wrote:
> Well, well - remarkable!!! an actual debate on connectionists - just
> like the old days - in fact REMARKABLY like the old days.
>
>
>
> Same issues - how ‘brain-like’ is ‘brain-like’ and how much hype is
> ‘brain-like’ generating by itself. How much do engineers really know
> about neuroscience, and how much do neurobiologists really know about
> the brain (both groups tend to claim they know a lot - now and then).
>
>
> I went to the NIPS meeting this year for the first time in more than
> 25 years. Some of the older timers on connectionists may remember
> that I was one of the founding members of NIPS - and some will also
> remember that a few years of trying to get some kind of real
> interaction between neuroscience and then ‘neural networks’ lead me to
> give up and start, with John Miller, the CNS meetings - focused
> specifically on computational neuroscience. Another story -
>
>
> At NIPS this year, there was a very large focus on “big data” of
> course, with "machine learning" largely replaced "Neural Networks" in
> most talk titles. I was actually a panelist (most had no idea of my
> early involvement with NIPS) on big data in on-line learning
> (generated by Ed-X, Kahn, etc) workshop. I was interested, because
> for 15 years I have also been running Numedeon Inc, whose virtual
> world for kids, Whyville.net was the first game based immersive
> worlds, and is still one of the biggest and most innovative. (no
> MOOCs there).
>
>
> From the panel I made the assertion, as I had, in effect, many years
> ago, that if you have a big data problem - it is likely you are not
> taking anything resembling a ‘brain-like’ approach to solving it. The
> version almost 30 years ago, when everyone was convinced that the
> relatively simple Hopfield Network could solve all kinds of hard
> problems, was my assertion that, in fact, simple ‘Neural Networks, or
> simple Neural Network learning rules were unlikely to work very well,
> because, almost certainly, you have to build a great deal of knowledge
> about the nature of the problem into all levels (including the input
> layer) of your network to get it to work.
>
>
> Now, many years later, everyone seems convinced that you can figure
> things out by amassing an enormous amount of data and working on it.
>
>
> It has been a slow revolution (may actually not even be at the
> revolutionary stage yet), BUT it is very likely that the nervous
> system (like all model based systems) doesn’t collect tons of data to
> figure out with feedforward processing and filtering, but instead,
> collects the data it thinks it needs to confirm what it already
> believes to be true. In other words, it specifically avoids the big
> data problem at all cost. It is willing to suffer the consequence
> that occasionally (more and more recently for me), you end up talking
> to someone for 15 minutes before you realize that they are not the
> person you thought they were.
>
>
> An enormous amount of engineering and neuroscience continues to think
> that the feedforward pathway is from the sensors to the inside -
> rather than seeing this as the actual feedback loop. Might to some
> sound like a semantic quibble, but I assure you it is not.
>
>
> If you believe as I do, that the brain solves very hard problems, in
> very sophisticated ways, that involve, in some sense the construction
> of complex models about the world and how it operates in the world,
> and that those models are manifest in the complex architecture of the
> brain - then simplified solutions are missing the point.
>
>
> What that means inevitably, in my view, is that the only way we will
> ever understand what brain-like is, is to pay tremendous attention
> experimentally and in our models to the actual detailed anatomy and
> physiology of the brains circuits and cells.
>
>
> I saw none of that at NIPS - and in fact, I see less and less of that
> at the CNS meeting as well.
>
>
> All too easy to simplify, pontificate, and sell.
>
>
> So, I sympathize with Juyang Wang’s frustration.
>
>
> If there is any better evidence that we are still in the dark, it is
> that we are still having the same debate 30 years later, with the same
> ruffled feathers, the same bold assertions (mine included) and the
> same seeming lack of progress.
>
>
> If anyone is interested, here is a chapter I recently wrote of the
> book I edited on “20 years of progress in computational neuroscience
> (Springer) on the last 40 years trying to understand the workings of a
> single neuron (The cerebellar Purkinje cell), using models.
> https://www.dropbox.com/s/5xxut90h65x4ifx/272602_1_En_5_DeltaPDF%
> 20copy.pdf
>
>
> Perhaps some sense of how far we have yet to go.
>
>
> Jim Bower
>
>
>
>
>
>
>
>
>
>
> On Jan 24, 2014, at 4:00 PM, Ralph Etienne-Cummings
> <ralph.etiennecummings at gmail.com> wrote:
>
>
>
> > Hey, I am happy when our taxpayer money, of which I contribute way
> > more than I get back, funds any science in all branches of the
> > government.
> >
> > Neuromorphic and brain-like computing is on the rise ... Let's
> > please not shoot ourselves in the foot with in-fighting!!
> >
> > Thanks,
> > Ralph's Android
> >
> >
> > On Jan 24, 2014 4:13 PM, "Juyang Weng" <weng at cse.msu.edu> wrote:
> >
> > Yes, Gary, you are correct politically, not to upset the
> > "emperor" since he is always right and he never falls behind
> > the literature.
> >
> > But then no clear message can ever get across. Falling
> > behind the literature is still the fact. More, the entire
> > research community that does brain research falls behind
> > badly the literature of necessary disciplines. The current
> > U.S. infrastructure of this research community does not fit
> > at all the brain subject it studies! This is not a joking
> > matter. We need to wake up, please.
> >
> > Azriel Rosenfeld criticized the entire computer vision filed
> > in his invited talk at CVPR during early 1980s: "just doing
> > business as usual" and "more or less the same" . However,
> > the entire computer vision field still has not woken up
> > after 30 years! As another example, I respect your
> > colleague Terry Sejnowski, but I must openly say that I
> > object to his "we need more data" as the key message for the
> > U.S. BRAIN Project. This is another example of "just doing
> > business as usual" and so everybody will not be against
> > you.
> >
> > Several major disciplines are closely related to the brain,
> > but the scientific community is still very much fragmented,
> > not willing to wake up. Some of our government officials
> > only say superficial worlds like "Big Data" because we like
> > to hear. This cost is too high for our taxpayers.
> >
> > -John
> >
> >
> > On 1/24/14 2:19 PM, Gary Cottrell wrote:
> >
> > >
> > > Hi John -
> > >
> > >
> > >
> > > It's great that you have an over-arching theory, but if
> > > you want people to read it, it would be better not to
> > > disrespect people in your emails. You say you respect
> > > Matthew, but then you accuse him of falling behind in the
> > > literature because he hasn't read your book. Politeness
> > > (and modesty!) will get you much farther than the tone you
> > > have taken.
> > >
> > >
> > > g.
> > >
> > >
> > > On Jan 24, 2014, at 6:27 PM, Juyang Weng
> > > <weng at cse.msu.edu> wrote:
> > >
> > >
> > >
> > > > Dear Matthew:
> > > >
> > > > My apology if my words are direct, so that people with
> > > > short attention spans can quickly get my points. I do
> > > > respect you.
> > > >
> > > > You wrote: "to build hardware that works in a more
> > > > brain-like way than conventional computers do. This is
> > > > not what is usually meant by research in neural
> > > > networks."
> > > >
> > > > Your statement is absolutely not true. Your term
> > > > "brain-like way" is as old as "brain-like computing".
> > > > Read about the 14 neurocomputers built by 1988 in Robert
> > > > Hecht-Nielsen, "Neurocomputing: picking the human
> > > > brain", IEEE Spectrum 25(3), March 1988, pp. 36-41.
> > > > Hardware will not solve the fundamental problems of the
> > > > current human severe lack in understanding the brain, no
> > > > matter how many computers are linked together. Neither
> > > > will the current "Big Data" fanfare from NSF in U.S..
> > > > The IBM's brain project has similar fundamental flaws
> > > > and the IBM team lacks key experts.
> > > >
> > > > Some of the NSF managers have been turning blind eyes to
> > > > breakthrough work on brain modeling for over a decade,
> > > > but they want to waste more taxpayer's money into its
> > > > "Big Data" fanfare and other "try again" fanfares. It
> > > > is a scientific shame for NSF in a developed country
> > > > like U.S. to do that shameful politics without real
> > > > science, causing another large developing country like
> > > > China to also echo "Big Data". "Big Data" was called
> > > > "Large Data", well known in Pattern Recognition for many
> > > > years. Stop playing shameful politics in science!
> > > >
> > > > You wrote: "Nobody is claiming a `brain-scale theory
> > > > that bridges the wide gap,' or even close."
> > > >
> > > > To say that, you have not read the book: Natural and
> > > > Artificial Intelligence. You are falling behind the
> > > > literature so bad as some of our NSF project managers.
> > > > With their lack of knowledge, they did not understand
> > > > that the "bridge" was in print on their desks and in the
> > > > literature.
> > > >
> > > > -John
> > > >
> > > >
> > > > On 1/23/14 6:15 PM, Matthew Cook wrote:
> > > >
> > > > >
> > > > > Dear John,
> > > > >
> > > > >
> > > > >
> > > > > I think all of us on this list are interested in
> > > > > brain-like computing, so I don't understand your
> > > > > negativity on the topic.
> > > > >
> > > > >
> > > > > Many of the speakers are involved in efforts to build
> > > > > hardware that works in a more brain-like way than
> > > > > conventional computers do. This is not what is
> > > > > usually meant by research in neural networks. I
> > > > > suspect the phrase "brain-like computing" is intended
> > > > > as an umbrella term that can cover all of these
> > > > > efforts.
> > > > >
> > > > >
> > > > > I think you are reading far more into the announcement
> > > > > than is there. Nobody is claiming a "brain-scale
> > > > > theory that bridges the wide gap," or even close. To
> > > > > the contrary, the announcement is very cautious,
> > > > > saying that intense research is "gradually increasing
> > > > > our understanding" and "beginning to shed light on the
> > > > > human brain". In other words, the research advances
> > > > > slowly, and we are at the beginning. There is
> > > > > certainly no claim that any of the speakers has
> > > > > finished the job.
> > > > >
> > > > >
> > > > > Similarly, the announcement refers to "successful
> > > > > demonstration of some of the underlying principles [of
> > > > > the brain] in software and hardware", which implicitly
> > > > > acknowledges that we do not have all the principles.
> > > > > There is nothing like a claim that anyone has enough
> > > > > principles to "explain highly integrated brain
> > > > > functions".
> > > > >
> > > > >
> > > > > You are concerned that this workshop will avoid the
> > > > > essential issue of the wide gap between neuron-like
> > > > > computing and highly integrated brain functions. What
> > > > > makes you think it will avoid this? We are all
> > > > > interested in filling this gap, and the speakers
> > > > > (well, the ones who I know) all either work on this,
> > > > > or work on supporting people who work on this, or
> > > > > both.
> > > > >
> > > > >
> > > > > This looks like it will be a very nice workshop, with
> > > > > talks from leaders in the field on a variety of
> > > > > topics, and I wish I were able to attend it.
> > > > >
> > > > >
> > > > > Matthew
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Jan 23, 2014, at 7:08 PM, Juyang Weng wrote:
> > > > >
> > > > >
> > > > >
> > > > > > Dear Anders,
> > > > > >
> > > > > > Interesting topic about the brain! But Brain-Like
> > > > > > Computing is misleading because neural networks have
> > > > > > been around for at least 70 years.
> > > > > >
> > > > > > I quote: "We are now approaching the point when our
> > > > > > knowledge will enable successful demonstrations of
> > > > > > some of the underlying principles in software and
> > > > > > hardware, i.e. brain-like computing."
> > > > > >
> > > > > > What are the underlying principles? I am concerned
> > > > > > that projects like "Brain-Like Computing" avoid
> > > > > > essential issues:
> > > > > > the wide gap between neuron-like computing and
> > > > > > well-known highly integrated brain functions.
> > > > > > Continuing this avoidance would again create bad
> > > > > > names for "brain-like computing", just such
> > > > > > behaviors did for "neural networks".
> > > > > >
> > > > > > Henry Markram criticized IBM's brain project which
> > > > > > does miss essential brain principles, but has he
> > > > > > published such principles?
> > > > > > Modeling individual neurons more and more precisely
> > > > > > will explain highly integrated brain functions?
> > > > > > From what I know, definitely not, by far.
> > > > > >
> > > > > > Has any of your 10 speakers published any
> > > > > > brain-scale theory that bridges the wide gap? Are
> > > > > > you aware of any such published theories?
> > > > > >
> > > > > > I am sorry for giving a CC to the list, but many on
> > > > > > the list said that they like to hear discussions
> > > > > > instead of just event announcements.
> > > > > >
> > > > > > -John
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 1/13/14 12:14 PM, Anders Lansner wrote:
> > > > > >
> > > > > > > Workshop on Brain-Like Computing, February 5-6
> > > > > > > 2014
> > > > > > >
> > > > > > > The exciting prospects of developing brain-like
> > > > > > > information processing is one of the Deans Forum
> > > > > > > focus areas.
> > > > > > > As a means to encourage progress in this research
> > > > > > > area a Workshop is arranged February 5th-6th 2014
> > > > > > > on KTH campus in Stockholm.
> > > > > > >
> > > > > > > The human brain excels over contemporary computers
> > > > > > > and robots in processing real-time unstructured
> > > > > > > information and uncertain data as well as in
> > > > > > > controlling a complex mechanical platform with
> > > > > > > multiple degrees of freedom like the human body.
> > > > > > > Intense experimental research complemented by
> > > > > > > computational and informatics efforts are
> > > > > > > gradually increasing our understanding of
> > > > > > > underlying processes and mechanisms in small
> > > > > > > animal and mammalian brains and are beginning to
> > > > > > > shed light on the human brain. We are now
> > > > > > > approaching the point when our knowledge will
> > > > > > > enable successful demonstrations of some of the
> > > > > > > underlying principles in software and hardware,
> > > > > > > i.e. brain-like computing.
> > > > > > >
> > > > > > > This workshop assembles experts, from the partners
> > > > > > > and also other leading names in the field, to
> > > > > > > provide an overview of the state-of-the-art in
> > > > > > > theoretical, software, and hardware aspects of
> > > > > > > brain-like computing.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > List of speakers
> > > > > > >
> > > > > > > Speaker
> > > > > > >
> > > > > > >
> > > > > > > Affiliation
> > > > > > >
> > > > > > >
> > > > > > > Giacomo Indiveri
> > > > > > >
> > > > > > >
> > > > > > > ETH Zürich
> > > > > > >
> > > > > > >
> > > > > > > Abigail Morrison
> > > > > > >
> > > > > > >
> > > > > > > Forschungszentrum Jülich
> > > > > > >
> > > > > > >
> > > > > > > Mark Ritter
> > > > > > >
> > > > > > >
> > > > > > > IBM Watson Research
> > > > > > > Center
> > > > > > >
> > > > > > >
> > > > > > > Guillermo Cecchi
> > > > > > >
> > > > > > >
> > > > > > > IBM Watson Research
> > > > > > > Center
> > > > > > >
> > > > > > >
> > > > > > > Anders Lansner
> > > > > > >
> > > > > > >
> > > > > > > KTH Royal Institute of
> > > > > > > Technology
> > > > > > >
> > > > > > >
> > > > > > > Ahmed Hemani
> > > > > > >
> > > > > > >
> > > > > > > KTH Royal Institute of
> > > > > > > Technology
> > > > > > >
> > > > > > >
> > > > > > > Steve Furber
> > > > > > >
> > > > > > >
> > > > > > > University of Manchester
> > > > > > >
> > > > > > >
> > > > > > > Kazuyuki Aihara
> > > > > > >
> > > > > > >
> > > > > > > University of Tokyo
> > > > > > >
> > > > > > >
> > > > > > > Karlheinz Meier
> > > > > > >
> > > > > > >
> > > > > > > Heidelberg University
> > > > > > >
> > > > > > >
> > > > > > > Andreas Schierwagen
> > > > > > >
> > > > > > >
> > > > > > > Leipzig University
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > For signing up to the Workshop please use the
> > > > > > > registration form found at http://bit.ly/1dkuBgR
> > > > > > >
> > > > > > > You need to sign up before January 28th.
> > > > > > >
> > > > > > > Web page:
> > > > > > > http://www.kth.se/en/om/internationellt/university-networks/deans-forum/workshop-on-brain-like-computing-1.442038
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > ******************************************
> > > > > > >
> > > > > > > Anders Lansner
> > > > > > >
> > > > > > > Professor in Computer Science, Computational
> > > > > > > biology
> > > > > > >
> > > > > > > School of Computer Science and Communication
> > > > > > >
> > > > > > > Stockholm University and Royal Institute of
> > > > > > > Technology (KTH)
> > > > > > >
> > > > > > > ala at kth.se, +46-70-2166122
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > __________________________________________________
> > > > > > >
> > > > > > > Detta epostmeddelande
> > > > > > > innehåller inget virus
> > > > > > > eller annan skadlig kod
> > > > > > > för avast! Antivirus är
> > > > > > > aktivt.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > --
> > > > > > Juyang (John) Weng, Professor
> > > > > > Department of Computer Science and Engineering
> > > > > > MSU Cognitive Science Program and MSU Neuroscience Program
> > > > > > 428 S Shaw Ln Rm 3115
> > > > > > Michigan State University
> > > > > > East Lansing, MI 48824 USA
> > > > > > Tel: 517-353-4388
> > > > > > Fax: 517-432-1061
> > > > > > Email: weng at cse.msu.edu
> > > > > > URL: http://www.cse.msu.edu/~weng/
> > > > > > ----------------------------------------------
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > --
> > > > Juyang (John) Weng, Professor
> > > > Department of Computer Science and Engineering
> > > > MSU Cognitive Science Program and MSU Neuroscience Program
> > > > 428 S Shaw Ln Rm 3115
> > > > Michigan State University
> > > > East Lansing, MI 48824 USA
> > > > Tel: 517-353-4388
> > > > Fax: 517-432-1061
> > > > Email: weng at cse.msu.edu
> > > > URL: http://www.cse.msu.edu/~weng/
> > > > ----------------------------------------------
> > > >
> > >
> > >
> > >
> > > [I am in Dijon, France on sabbatical this year. To call
> > > me, Skype works best (gwcottrell), or dial +33 788319271]
> > >
> > >
> > > Gary Cottrell 858-534-6640 FAX: 858-534-7029
> > >
> > >
> > > My schedule is here: http://tinyurl.com/b7gxpwo
> > >
> > > Computer Science and Engineering 0404
> > > IF USING FED EX INCLUDE THE FOLLOWING LINE:
> > > CSE Building, Room 4130
> > > University of California San Diego
> > > 9500 Gilman Drive # 0404
> > > La Jolla, Ca. 92093-0404
> > >
> > >
> > > Things may come to those who wait, but only the things
> > > left by those who hustle. -- Abraham Lincoln
> > >
> > >
> > > "Of course, none of this will be easy. If it was, we would
> > > already know everything there was about how the brain
> > > works, and presumably my life would be simpler here. It
> > > could explain all kinds of things that go on
> > > in Washington." -Barack Obama
> > >
> > >
> > > "Probably once or twice a week we are sitting at dinner
> > > and Richard says, 'The cortex is hopeless,' and I say,
> > > 'That's why I work on the worm.'" Dr. Bargmann said.
> > >
> > > "A grapefruit is a lemon that saw an opportunity and took
> > > advantage of it." - note written on a door in Amsterdam on
> > > Lijnbaansgracht.
> > >
> > > "Physical reality is great, but it has a lousy search
> > > function." -Matt Tong
> > >
> > > "Only connect!" -E.M. Forster
> > >
> > > "You always have to believe that tomorrow you might write
> > > the matlab program that solves everything - otherwise you
> > > never will." -Geoff Hinton
> > >
> > >
> > > "There is nothing objective about objective functions" -
> > > Jay McClelland
> > >
> > > "I am awaiting the day when people remember the fact that
> > > discovery does not work by deciding what you want and then
> > > discovering it."
> > > -David Mermin
> > >
> > > Email: gary at ucsd.edu
> > > Home page: http://www-cse.ucsd.edu/~gary/
> > >
> > >
> > >
> >
> >
> >
> > --
> > --
> > Juyang (John) Weng, Professor
> > Department of Computer Science and Engineering
> > MSU Cognitive Science Program and MSU Neuroscience Program
> > 428 S Shaw Ln Rm 3115
> > Michigan State University
> > East Lansing, MI 48824 USA
> > Tel: 517-353-4388
> > Fax: 517-432-1061
> > Email: weng at cse.msu.edu
> > URL: http://www.cse.msu.edu/~weng/
> > ----------------------------------------------
> >
>
>
>
>
>
>
>
> Dr. James M. Bower Ph.D.
>
> Professor of Computational Neurobiology
>
> Barshop Institute for Longevity and Aging Studies.
>
> 15355 Lambda Drive
>
> University of Texas Health Science Center
>
> San Antonio, Texas 78245
>
>
>
> Phone: 210 382 0553
>
> Email: bower at uthscsa.edu
>
> Web: http://www.bower-lab.org
>
> twitter: superid101
>
> linkedin: Jim Bower
>
>
>
> CONFIDENTIAL NOTICE:
>
> The contents of this email and any attachments to it may be privileged
> or contain privileged and confidential information. This information
> is only for the viewing or use of the intended recipient. If you have
> received this e-mail in error or are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution or use
> of, or the taking of any action in reliance upon, any of the
> information contained in this e-mail, or
>
> any of the attachments to this e-mail, is strictly prohibited and that
> this e-mail and all of the attachments to this e-mail, if any, must be
>
> immediately returned to the sender or destroyed and, in either case,
> this e-mail and all attachments to this e-mail must be immediately
> deleted from your computer without making any copies hereof and any
> and all hard copies made must be destroyed. If you have received this
> e-mail in error, please notify the sender by e-mail immediately.
>
>
>
>
>
>
>
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140125/0bd99183/attachment.html>
More information about the Connectionists
mailing list