Connectionists: Brain-like computing fanfare and big data fanfare

[Brian J Mingus]

Hi Brad et al.,  - thanks very much for this fun and entertaining
philosophical discussion:)

With regards to turtles all the way down, and also with regards to choosing
the appropriate level of analysis for modeling, I'd like to reiterate a
position I made earlier but  in which I didn't really provide enough detail.

There exists a formalization of Ockham's razor in a field called
Algorithmic Information Theory, and this formalization is the Minimum
Description Length (MDL).

This perspective essentially says that we are searching for the optimal
compression of all of the data relating to the brain. This means that we
don't want to overcompress relevant distinctions, but we don't want to
undercompress redundancies. This optimal compression, when represented as a
computer program that outputs all of the brain data (aka a model), has a
description length known as the Kolmogorov complexity.

Now there is something weird about what I have just described, which is
that the resulting model will produce not just the data for a single brain,
but the data for *every* brain - a kind of meta-brain. And this is not
quite what we are looking for. And due to the turtles problem it is
probably ill-posed, in that the length of the description may be infinite
as we zoom in to finer levels of detail.

So we need to provide some relevant constraints on the problem to make it
tractable. Based on what I just described, the MDL for your brain *is* your
brain. This is essentially because we haven't defined a utility function,
and we haven't done that because we aren't quite sure what exactly it is we
are doing, or what we are looking for, when modeling the brain.

To begin fixing this problem, we can rotate this perspective into a tool
that we are all probably familiar with - factor analysis, i.e., PCA. What
we are essentially looking for, first and foremost, is a model that
explains the first principle component of just one person's comprehensive
brain dataset (which includes behavioral data). Then we want to study this
component (which is tantamount to a model of the brain) and see what it can
do.

What will this first principle component look like? Now we need to define
what exactly it is that we are after. I would argue that our model should
be composed of neuron-like elements connected in networks, and that when we
look at the statistical properties of these networks, they should be quite
similar to what we see in humans.

Most importantly, however, I would argue that this model, when raised as a
human, should exhibit some distinctly human traits. It should not pass a
trivial turing test, but rather a deep turing test. After having been
raised as and with human beings, but not exposed to any substantial
philosophy, this model should independently invent consciousness philosophy.

As you might imagine, our abstract high level model brain which captures
the first principle component of the brain data might not be able to do
this. Thus, we will start adding in more components that explain more of
the variance, iteratively increasing our description length. This is a
distinctly top-down approach, in which we only add relevant detail as it
becomes obvious that the current model just isn't quite human.

This approach follows a scientific gradient advocated for by Ockham's
razor, in that we start with the simplest description (brain model) that
explains the most amount of variance, and gradually increase the size of
the description until it finally reinvents consciousness philosophy and can
live among humans.

In my admittedly biased experience, the first appropriate level of analysis
is approximately point-neuron deep neural network architectures. However,
this might actually be too low level - we might want to start with even
more abstract, modern day NIPS-level models, and confirm that, although
they can behave like humans, they can't reinvent consciousness philosophy
and are thus more akin to zombie-like automata.

Of course, with sufficient computing power our modeling approach can be
somewhat more sloppy - we can begin experimenting with the synthesis of
different levels of analysis right away.

However, before we do any of this "for real" we probably want to
comprehensively discuss the ethics of raising beings that are ultimately
similar to humans, but are not quite human, and further, the ethics of
raising digital humans.

Lastly, to touch back to the original topic - Big Data - I think it's clear
that the more data we have, the merrier. However, it also makes sense to
follow the Ockham gradient. Ultimately, we are really just not as close to
creating a human being as it may seem, and so it is probably safe, for the
time being, to collect data from all levels of analysis willy nilly.
However, when it comes time to actually build the human, we should be more
careful, for the sake of the being we create. Indeed, perhaps we should be
*sure* that it will reinvent consciousness philosophy before we ever turn
it on in the first place.

If anyone has an idea of how to do that, I would be extremely interested to
hear about it.

Brian Mingus

Graduate student
Department of Psychology and Neuroscience
University of Colorado at Boulder
http://grey.colorado.edu/mingus


On Sat, Jan 25, 2014 at 7:52 PM, Brad Wyble <bwyble at gmail.com> wrote:

> Jim,
>
> Great debate!  There are several good points here..
>
> First, I agree with you that models with tidy, analytical solutions are
> probably not the ultimate answer, as biology is unlikely to exhibit
> behavior that coincides with mathematical formalisms that are easy to
> represent in equations. In fact, I think that seeking such solutions can
> get in the way of progress in some cases.
>
> I also agree with you that community models are a good idea, and I am not
> advocating that everyone should build their own model.  But I think that we
> need a hierarchy of such community models at multiple levels of
> abstraction, with clear ways of translating ideas and constraints from each
> level to the next.  The goal of computational neuroscience is not to build
> the ultimate model, but to build a shared understanding in the minds of the
> entire body of neuroscientists with a minimum of communication failures.
>
> Next,  I think that you're espousing a purely bottom-up approach to
> modelling the brain. ( i.e. that if we just build it, understanding will
> follow from the emergent dynamics). I very much admire your strong
> position, but I really can't agree with it.  I return to the question of
> how we will even know what the bottom floor is in such an approach  You
> seem to imply in previous emails that it's a channel/cable model, but
> someone else might argue that we'd have to represent interactions at the
> atomic level to truly capture the dynamics of the circuit.  So if that's
> the only place to start, how will we ever make serious progress?  The
> computational requirements to simulate even a single neuron at the atomic
> level on a super cluster is probably a decade away. And once we'd
> accomplished that, someone might point out a case in which subatomic
> interactions play a functional role in the neuron and then we've got to
> wait another 10 years to be able to model a single neuron again?
>
> To me, it really looks like turtles all the way down which means that we
> have to choose our levels of abstraction with an understanding that there
> are important dynamics at lower levels that will be missed.  However if we
> build in constraints from the behavior of the system, such abstract models
> can nevertheless provide a foothold for climbing a bit higher in our
> understanding.
>
> Is there some reason that you think channels are a sufficient level of
> detail?  (or maybe I've mischaracterized your position)
>
> -Brad
>
>
>
>
>
> On Sat, Jan 25, 2014 at 7:09 PM, james bower <bower at uthscsa.edu> wrote:
>
>> About to sign off here - as have probably already taken too much
>> bandwidth. (although it has been a long time)
>>
>> But just for final clarity on the point about physics - I am not claiming
>> that the actual tools etc, developed by physics mostly to study
>> non-biological and mostly ‘simpler’ systems (for example, systems were the
>> elements (unlike neurons) aren’t ‘individualized’  - and therefore can be
>> subjected to a certain amount of averaging (ie. thermodynamics), will apply.
>>
>> But I am suggesting (all be it in an oversimplified way)  that the
>> transition from a largely folkloric, philosophically (religiously) driven
>> style of physics, to the physics of today was accomplished in the 15th
>> century by the rejection of the curve fitting, ‘simplified’ and self
>> reflective Ptolemic model of the solar system. (not actually, it turns out
>> for that reason, but because the Ptolemaic model has become too complex and
>> impure - the famous equint point).   Instead, Newton, Kepler, etc, further
>> developed a model that actually valued the physical structure of that
>> system, independent of the philosophical, self reflecting previous set of
>> assumptions.  I know, I know that this is an oversimplified description of
>> what happened, but, it is very likely that Newtons early (age 19) discovery
>> of what approximated the least squares law in the ‘realistic model’ he had
>> constructed of the earth moon system (where it was no problem and pretty
>> clearly evident that the moon orbited the earth in a regular way), lead in
>> later years to his development of mechanics - which, clearly provided an
>> important "community model” of the sort we completely  lack in neuroscience
>> and seem to me continue to try to avoid.
>>
>> I have offered for years to buy the beer at the CNS meeting if all the
>> laboratories describing yet another model of the hippocampus or the visual
>> cortex would get together to agree on a single model they would all work
>> on.  No takers yet.  The paper I linked to in my first post describes how
>> that has happened for the Cerebellar Purkinje cell, because of GENESIS and
>> because we didn’t block others from using the model, even to criticize us.
>>    However, when I sent that paper recently to a computational neuroscience
>> I heard was getting into Purkinje cell modeling, he wrote back to say he
>> was developing his own model thank you very much.
>>
>> The proposal that we all be free to build our own models - and everyone
>> is welcome, is EXACTLY the wrong direction.
>>
>> We need more than calculous - and although I understand their
>> attractiveness believe me, models that can be solved in close formed
>> solutions are not likely to be particularly useful in biology, where the
>> averaging won’t work in the same way. The relationship between scales is
>> different, lots of things are different - which means the a lot of the
>> tools will have to be different too. And I even agree that some of the
>> tools developed by engineering, where one is actually trying to make things
>> that work, might end up being useful, or even perhaps more useful.
>>  However, the transition to paradigmatic science I believe will critically
>> depend on the acceptance of community models (they are the ‘paradigm’), and
>> the models most likely with the most persuasive force as well as the ones
>> mostly likelihood of revealing unexpected functional relationships, are
>> ones that FIRST account for the structure of the brain, and SECOND are used
>> to explore function (rather than what is usually the other way around).
>>
>> As described in the paper I posted, that is exactly what has happened
>> through long hard work (since 1989) using the Purkinje cell model.
>>
>> In the end, unless you are a duelist (which I suspect many actually are,
>> in effect), brain computation involves nothing beyond the nervous system
>> and its physical and physiological structure.  Therefore, that structure
>> will be the ultimate reference for how things really work, no matter what
>> level of scale you seek to describe.
>>
>> From 30 years of effort, I believe even more firmly now than I did back
>> then, that, like Newton and his friends, this is where we should start -
>> figuring out the principles and behavior from the physics of the elements
>> themselves.
>>
>> You can claim it is impossible - you can claim that models at other
>> levels of abstraction can help, however, in the end ‘the truth’ lies in the
>> circuitry in all its complexity.  But you can’t just jump into the
>> complexity, without a synergistic link to models that actually provide
>> insights at the detailed level of the data you seek to collect.
>>
>> IMHO.
>>
>> Jim
>>
>> (no ps)
>>
>>
>>
>>
>>
>>
>>
>> On Jan 25, 2014, at 4:44 PM, Dan Goodman <
>> dg.connectionists at thesamovar.net> wrote:
>>
>> The comparison with physics is an interesting one, but we have to
>> remember that neuroscience isn't physics. For a start, neuroscience is
>> clearly much harder than physics in many ways. Linear and separable
>> phenomena are much harder to find in neuroscience, and so both analysing
>> and modelling data is much more difficult. Experimentally, it is much more
>> difficult to control for independent variables in addition to the
>> difficulty of working with living animals.
>>
>> So although we might be able to learn things from the history of physics
>> - and I tend to agree with Axel Hutt that one of those lessons is to use
>> the simplest possible model rather than trying to include all the
>> biophysical details we know to exist - while neuroscience is in its
>> pre-paradigmatic phase (agreed with Jim Bower on this) I would say we need
>> to try a diverse set of methodological approaches and see what wins. In
>> terms of funding agencies, I think the best thing they could do would be to
>> not insist on any one methodological approach to the exclusion of others.
>>
>> I also share doubts about the idea that if we collect enough data then
>> interesting results will just pop out. On the other hand, there are some
>> valid hypotheses about brain function that require the collection of large
>> amounts of data. Personally, I think that we need to understand the
>> coordinated behaviour of many neurons to understand how information is
>> encoded and processed in the brain. At present, it's hard to look at enough
>> neurons simultaneously to be very sure of finding this sort of coordinated
>> activity, and this is one of the things that the HBP and BRAIN initiative
>> are aiming at.
>>
>> Dan
>>
>>
>>
>>
>>
>>
>> Dr. James M. Bower Ph.D.
>>
>> Professor of Computational Neurobiology
>>
>> Barshop Institute for Longevity and Aging Studies.
>>
>> 15355 Lambda Drive
>>
>> University of Texas Health Science Center
>>
>> San Antonio, Texas  78245
>>
>>
>>
>> *Phone:  210 382 0553 <210%20382%200553>*
>>
>> Email: bower at uthscsa.edu
>>
>> Web: http://www.bower-lab.org
>>
>> twitter: superid101
>>
>> linkedin: Jim Bower
>>
>>
>>
>> CONFIDENTIAL NOTICE:
>>
>> The contents of this email and any attachments to it may be privileged
>> or contain privileged and confidential information. This information is
>> only for the viewing or use of the intended recipient. If you have received
>> this e-mail in error or are not the intended recipient, you are hereby
>> notified that any disclosure, copying, distribution or use of, or the
>> taking of any action in reliance upon, any of the information contained in
>> this e-mail, or
>>
>> any of the attachments to this e-mail, is strictly prohibited and that
>> this e-mail and all of the attachments to this e-mail, if any, must be
>>
>> immediately returned to the sender or destroyed and, in either case,
>> this e-mail and all attachments to this e-mail must be immediately deleted
>> from your computer without making any copies hereof and any and all hard
>> copies made must be destroyed. If you have received this e-mail in error,
>> please notify the sender by e-mail immediately.
>>
>>
>>
>>
>
>
> --
> Brad Wyble
> Assistant Professor
> Psychology Department
> Penn State University
>
> http://wyblelab.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140125/46648e07/attachment.html>