Connectionists: Brain-like computing fanfare and big data fanfare

Fri Jan 24 16:38:03 EST 2014

Personally, I think Big Data is a great thing, and I think we need
well-funded people working at all levels of analysis. For example, Big Data
will allow us to boot up a rather high fidelity brain on a supercomputer.
This will help us determine the extent to which we can compress that
representation into a simpler model that can perform the same essential
computations. We can probably compress much of what the brain does into
higher level models, however, you can implement all sorts of clever
machinery using neurons, and we may find that certain things (such as the
intricate circuitry of the cerebellum) should not be compressed too much.
Unless it's a Kalman filter, in which case we may choose to swap it out for
one.

Peter Norvig has co-authored a great paper called The Unreasonable
Effectiveness of Data which you can find here: http://goo.gl/klOZGA. It is
focused on the fact that when you have gobs of data the algorithm doesn't
matter as much, but I think it also goes to show that Big Data will make
figuring out how the brain works simpler.

Going further, having lots of data is a necessary requirement for
approximating the Minimum Description Length model of the brain. This MDL
model will compress irrelevant detail while leaving relevant detail intact.
In order to demonstrate that our MDL model generalizes to real brains,
we'll need as much brain data as we can get our hands on.
https://en.wikipedia.org/wiki/Minimum_description_length

We also need adequate funding for detailed single-neuron neurophysiological
models, middle-of-the-ground attractor networks, more abstract normalized
models and machine learning research in general, fMRI research, EEG
research, etc. None of these areas should be neglected, and they should all
be well-funded, as they all provide relevant constraints.

Practically speaking, funding is limited and those who approve grants tend
to use hyperbolic discounting just like the rest of us, so it can be
challenging to convince them to give you money despite your relevance to
our overall long-term goal, leading to frustration. I'd guess though that
we can all agree that all areas of brain research are extremely valuable
and underfunded. Luckily, people are starting to take note of how cool the
brain is, and we have a technological singularity on the horizon. If we
hold our breath, these problems are probably going to go away.

$.02

Brian Mingus

http://grey.colorado.edu/mingus

On Fri, Jan 24, 2014 at 1:24 PM, Juyang Weng <weng at cse.msu.edu> wrote:

>  Yes, Gary, you are correct politically, not to upset the "emperor" since
> he is always right and he never falls behind the literature.
>
> But then no clear message can ever get across.   Falling behind the
> literature is still the fact.  More, the entire research community that
> does brain research falls behind badly the literature of necessary
> disciplines.  The current U.S. infrastructure of this research community
> does not fit at all the brain subject it studies!  This is not a joking
> matter.  We need to wake up, please.
>
> Azriel Rosenfeld criticized the entire computer vision filed in his
> invited talk at CVPR during early 1980s: "just doing business as usual" and
> "more or less the same" .   However, the entire computer vision field still
> has not woken up after 30 years!   As another example, I respect your
> colleague Terry Sejnowski, but I must openly say that I object to his "we
> need more data" as the key message for the U.S. BRAIN Project.  This is
> another example of "just doing business as usual" and so everybody will not
> be against you.
>
> Several major disciplines are closely related to the brain, but the
> scientific community is still very much fragmented, not willing to wake
> up.  Some of our government officials only say superficial worlds like "Big
> Data" because we like to hear.   This cost is too high for our taxpayers.
>
> -John
>
> On 1/24/14 2:19 PM, Gary Cottrell wrote:
>
> Hi John -
>
>  It's great that you have an over-arching theory, but if you want people
> to read it, it would be better not to disrespect people in your emails. You
> say you respect Matthew, but then you accuse him of falling behind in the
> literature because he hasn't read your book. Politeness (and modesty!) will
> get you much farther than the tone you have taken.
>
>  g.
>
>  On Jan 24, 2014, at 6:27 PM, Juyang Weng <weng at cse.msu.edu> wrote:
>
>  Dear Matthew:
>
> My apology if my words are direct, so that people with short attention
> spans can quickly get my points.  I do respect you.
>
> You wrote: "to build hardware that works in a more brain-like way than
> conventional computers do.  This is not what is usually meant by research
> in neural networks."
>
> Your statement is absolutely not true.  Your term "brain-like way" is as
> old as "brain-like computing".  Read about the 14 neurocomputers built by
> 1988 in Robert Hecht-Nielsen, "Neurocomputing: picking the human brain",
> IEEE Spectrum 25(3), March 1988, pp. 36-41.  Hardware will not solve the
> fundamental problems of the current human severe lack in understanding the
> brain, no matter how many computers are linked together.  Neither will the
> current "Big Data" fanfare from NSF in U.S..  The IBM's brain project has
> similar fundamental flaws and the IBM team lacks key experts.
>
> Some of the NSF managers have been turning blind eyes to breakthrough work
> on brain modeling for over a decade, but they want to waste more taxpayer's
> money into its "Big Data" fanfare and other "try again" fanfares.  It is a
> scientific shame for NSF in a developed country like U.S. to do that
> shameful politics without real science, causing another large developing
> country like China to also echo "Big Data".  "Big Data" was called "Large
> Data", well known in Pattern Recognition for many years.  Stop playing
> shameful politics in science!
>
> You wrote: "Nobody is claiming a `brain-scale theory that bridges the wide
> gap,' or even close."
>
> To say that, you have not read the book: Natural and Artificial
> Intelligence <http://www.brain-mind-institute.org/press.html>.  You are
> falling behind the literature so bad as some of our NSF project managers.
> With their lack of knowledge, they did not understand that the "bridge" was
> in print on their desks and in the literature.
>
> -John
>
> On 1/23/14 6:15 PM, Matthew Cook wrote:
>
> Dear John,
>
>  I think all of us on this list are interested in brain-like computing,
> so I don't understand your negativity on the topic.
>
>  Many of the speakers are involved in efforts to build hardware that
> works in a more brain-like way than conventional computers do.  This is not
> what is usually meant by research in neural networks.  I suspect the phrase
> "brain-like computing" is intended as an umbrella term that can cover all
> of these efforts.
>
>  I think you are reading far more into the announcement than is there.
>  Nobody is claiming a "brain-scale theory that bridges the wide gap," or
> even close.  To the contrary, the announcement is very cautious, saying
> that intense research is "gradually increasing our understanding" and
> "beginning to shed light on the human brain".  In other words, the research
> advances slowly, and we are at the beginning.  There is certainly no claim
> that any of the speakers has finished the job.
>
>  Similarly, the announcement refers to "successful demonstration of some
> of the underlying principles [of the brain] in software and hardware",
> which implicitly acknowledges that we do not have all the principles.
>  There is nothing like a claim that anyone has enough principles to
> "explain highly integrated brain functions".
>
>  You are concerned that this workshop will avoid the essential issue of
> the wide gap between neuron-like computing and highly integrated brain
> functions.  What makes you think it will avoid this?  We are all interested
> in filling this gap, and the speakers (well, the ones who I know) all
> either work on this, or work on supporting people who work on this, or both.
>
>  This looks like it will be a very nice workshop, with talks from leaders
> in the field on a variety of topics, and I wish I were able to attend it.
>
>  Matthew
>
>
>  On Jan 23, 2014, at 7:08 PM, Juyang Weng wrote:
>
>  Dear Anders,
>
> Interesting topic about the brain!  But Brain-Like Computing is misleading
> because neural networks have been around for at least 70 years.
>
> I quote: "We are now approaching the point when our knowledge will enable
> successful demonstrations of some of the underlying principles in software
> and hardware, i.e. brain-like computing."
>
> What are the underlying principles?  I am concerned that projects like
> "Brain-Like Computing" avoid essential issues:
> the wide gap between neuron-like computing and well-known highly
> integrated brain functions.
> Continuing this avoidance would again create bad names for "brain-like
> computing", just such behaviors did for "neural networks".
>
> Henry Markram criticized IBM's brain project which does miss essential
> brain principles, but has he published such principles?
> Modeling individual neurons more and more precisely will explain highly
> integrated brain functions?   From what I know, definitely not, by far.
>
> Has any of your 10 speakers published any brain-scale theory that bridges
> the wide gap?  Are you aware of any such published theories?
>
> I am sorry for giving a CC to the list, but many on the list said that
> they like to hear discussions instead of just event announcements.
>
> -John
>
>
> On 1/13/14 12:14 PM, Anders Lansner wrote:
>
>  Workshop on Brain-Like Computing, February 5-6 2014
>
> The exciting prospects of developing brain-like information processing is
> one of the Deans Forum focus areas.
> As a means to encourage progress in this research area a Workshop is
> arranged February 5th-6th 2014 on KTH campus in Stockholm.
>
> The human brain excels over contemporary computers and robots in
> processing real-time unstructured information and uncertain data as well as
> in controlling a complex mechanical platform with multiple degrees of
> freedom like the human body. Intense experimental research complemented by
> computational and informatics efforts are gradually increasing our
> understanding of underlying processes and mechanisms in small animal and
> mammalian brains and are beginning to shed light on the human brain. We are
> now approaching the point when our knowledge will enable successful
> demonstrations of some of the underlying principles in software and
> hardware, i.e. brain-like computing.
>
> This workshop assembles experts, from the partners and also other leading
> names in the field, to provide an overview of the state-of-the-art in
> theoretical, software, and hardware aspects of brain-like computing.
> List of speakers
>
> *Speaker*
>
> *Affiliation*
>
> Giacomo Indiveri
>
> ETH Zürich
>
> Abigail Morrison
>
> Forschungszentrum Jülich
>
> Mark Ritter
>
> IBM Watson Research Center
>
> Guillermo Cecchi
>
> IBM Watson Research Center
>
> Anders Lansner
>
> KTH Royal Institute of Technology
>
> Ahmed Hemani
>
> KTH Royal Institute of Technology
>
> Steve Furber
>
> University of Manchester
>
> Kazuyuki Aihara
>
> University of Tokyo
>
> Karlheinz Meier
>
> Heidelberg University
>
> Andreas Schierwagen
>
> Leipzig University
>
>
>
> *For signing up to the Workshop please use the registration form found at
> http://bit.ly/1dkuBgR <http://bit.ly/1dkuBgR>*
>
> *You need to sign up before January 28th.*
>
> *Web page:
> http://www.kth.se/en/om/internationellt/university-networks/deans-forum/workshop-on-brain-like-computing-1.442038
> <http://www.kth.se/en/om/internationellt/university-networks/deans-forum/workshop-on-brain-like-computing-1.442038>
> *
>
>
>
>
>
>
>
> ******************************************
>
> Anders Lansner
>
> Professor in Computer Science, Computational biology
>
> School of Computer Science and Communication
>
> Stockholm University and Royal Institute of Technology (KTH)
>
> ala at kth.se, +46-70-2166122
>
>
>
>
> ------------------------------
>     <http://www.avast.com/>
>
> Detta epostmeddelande innehåller inget virus eller annan skadlig kod för avast!
> Antivirus <http://www.avast.com/> är aktivt.
>
>
> --
> --
> Juyang (John) Weng, Professor
> Department of Computer Science and Engineering
> MSU Cognitive Science Program and MSU Neuroscience Program
> 428 S Shaw Ln Rm 3115
> Michigan State University
> East Lansing, MI 48824 USA
> Tel: 517-353-4388
> Fax: 517-432-1061
> Email: weng at cse.msu.edu
> URL: http://www.cse.msu.edu/~weng/
> ----------------------------------------------
>
>
>
>
> --
> --
> Juyang (John) Weng, Professor
> Department of Computer Science and Engineering
> MSU Cognitive Science Program and MSU Neuroscience Program
> 428 S Shaw Ln Rm 3115
> Michigan State University
> East Lansing, MI 48824 USA
> Tel: 517-353-4388
> Fax: 517-432-1061
> Email: weng at cse.msu.edu
> URL: http://www.cse.msu.edu/~weng/
> ----------------------------------------------
>
>
>
>  [I am in Dijon, France on sabbatical this year. To call me, Skype works
> best (gwcottrell), or dial +33 788319271]
>
>  Gary Cottrell 858-534-6640 FAX: 858-534-7029
>
>  My schedule is here: http://tinyurl.com/b7gxpwo
>
> Computer Science and Engineering 0404
> IF USING FED EX INCLUDE THE FOLLOWING LINE:
> CSE Building, Room 4130
> University of California San Diego
> 9500 Gilman Drive # 0404
> La Jolla, Ca. 92093-0404
>
>  Things may come to those who wait, but only the things left by those who
> hustle. -- Abraham Lincoln
>
>  "Of course, none of this will be easy. If it was, we would already
> know everything there was about how the brain works, and presumably my
> life would be simpler here. It could explain all kinds of things that go on
> in Washington." -Barack Obama
>
>  "Probably once or twice a week we are sitting at dinner and Richard
> says, 'The cortex is hopeless,' and I say, 'That's why I work on the
> worm.'" Dr. Bargmann said.
>
> "A grapefruit is a lemon that saw an opportunity and took advantage of
> it." - note written on a door in Amsterdam on Lijnbaansgracht.
>
> "Physical reality is great, but it has a lousy search function." -Matt Tong
>
> "Only connect!" -E.M. Forster
>
> "You always have to believe that tomorrow you might write the matlab
> program that solves everything - otherwise you never will." -Geoff Hinton
>
>  "There is nothing objective about objective functions" - Jay McClelland
>
> "I am awaiting the day when people remember the fact that discovery does
> not work by deciding what you want and then discovering it."
> -David Mermin
>
> Email: gary at ucsd.edu
> Home page: http://www-cse.ucsd.edu/~gary/
>
>
> --
> --
> Juyang (John) Weng, Professor
> Department of Computer Science and Engineering
> MSU Cognitive Science Program and MSU Neuroscience Program
> 428 S Shaw Ln Rm 3115
> Michigan State University
> East Lansing, MI 48824 USA
> Tel: 517-353-4388
> Fax: 517-432-1061
> Email: weng at cse.msu.edu
> URL: http://www.cse.msu.edu/~weng/
> ----------------------------------------------
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140124/e4de3da0/attachment.html>