Connectionists: how the brain works?

[Tsvi Achler]

Basically it is a definition of over-fitting.  The fact large that models
may be needed to capture neuron components such proteins and so on do not
change the fact that too many extraneous parameters can make the model
precise but brittle.
-Tsvi


On Thu, Mar 20, 2014 at 10:59 AM, james bower <bower at uthscsa.edu> wrote:

> Interesting definition -
>
> just to note, we build realistic biological models with hundreds to
> thousands of parameters to model a neuron - however, they are not "free".
>
> in fact, it is easier to get a fake model of a neuron to behave like a
> neuron than it is to make one designed to replicate the anatomy and
> physiology.  Something that many in the physics / engineering worlds don't
> realize.  So, in fact, by the definition you use, abstract neuronal models
> which have a smaller number of essentially completely free parameters can't
> use Ockham's shaving cream as cover.  :-)
>
> Again, to return to my old analogy - Kepler was forced BY THE DATA to use
> ellipses for the orbits of the planets.  It is pretty clear he would rather
> have not.  Newton on the other hand clearly benefited from the fact that
> the moons orbit is essentially circular in his first calculation of the
> inverse square law.  In both cases, however, unlike Ptolomy, they were
> constructing physically realistic models.  Doing so in Biology also
> requires you deal with complexity you would rather not (as I keep saying ad
> naus......  )
>
> Jim
>
>
> On Mar 20, 2014, at 9:41 AM, Tsvi Achler <achler at gmail.com> wrote:
>
> > I think an Ockham's razor principle can be used to find the most
> > optimal algorithm if it is interpreted to mean the model with the
> > least amount of free parameters that captures the most phenomena.
> > http://reason.cs.uiuc.edu/tsvi/Evaluating_Flexibility_of_Recognition.pdf
> > -Tsvi
> >
> > On Wed, Mar 19, 2014 at 10:37 PM, Andras Lorincz <lorincz at inf.elte.hu>
> wrote:
> >> Ockham works here via compressing both the algorithm and the structure.
> >> Compressing the structure to stem cells means that the algorithm should
> >> describe the development, the working, and the time dependent structure
> of
> >> the brain. Not compressing the description of the structure of the
> evolved
> >> brain is a different problem since it saves the need for the
> description of
> >> the development, but the working. Understanding the structure and the
> >> working of one part of the brain requires the description of its
> >> communication that increases the complexity of the description. By the
> way,
> >> this holds for the whole brain, so we might have to include the body at
> >> least; a structural minimist may wish to start from the genetic code,
> use
> >> that hint and unfold the already compressed description. There are
> (many and
> >> different) todos 'outside' ...
> >>
> >>
> >> Andras
> >>
> >>
> >>
> >>
> >> .
> >>
> >> ________________________________
> >> From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on
> >> behalf of james bower <bower at uthscsa.edu>
> >> Sent: Thursday, March 20, 2014 3:33 AM
> >>
> >> To: Geoffrey Goodhill
> >> Cc: connectionists at mailman.srv.cs.cmu.edu
> >> Subject: Re: Connectionists: how the brain works?
> >>
> >> Geoffrey,
> >>
> >> Nice addition to the discussion actually introducing an interesting
> angle on
> >> the question of brain organization (see below)  As you note, reaction
> >> diffusion mechanisms and modeling have been quite successful in
> replicating
> >> patterns seen in biology - especially interesting I think is the
> modeling of
> >> patterns in slime molds, but also for very general pattern formation in
> >> embryology.  However, more and more detailed analysis of what is
> diffusing,
> >> what is sensing what is diffusing, and what is reacting to substances
> once
> >> sensed -- all linked to complex patterns of gene regulation and
> expression
> >> have made it clear that actual embryological development is much much
> more
> >> complex, as Turing himself clearly anticipated, as the quote you cite
> pretty
> >> clearly indicates.   Clearly a smart guy.   But, I don't actually think
> that
> >> this is an application of Ochham's razor although it might appear to be
> >> after the fact.  Just as Hodgkin and Huxley were not applying it either
> in
> >> their model of the action potential.   Turing apparently guessed (based
> on a
> >> lot of work at the time on pattern formation with reaction diffusion)
> that
> >> such a mechanism might provide the natural basis for what embryos do.
> Thus,
> >> just like for Hodgkin and Huxley, his model resulted from a bio-physical
> >> insight, not an explicit attempt to build a stripped down model for its
> own
> >> sake.  I  seriously doubt that Turning would have claimed that he, or
> his
> >> models could more effectively do what biology actually does in forming
> an
> >> embrio, or substitute for the actual process.
> >>
> >> However, I think there is another interesting connection here to the
> >> discussion on modeling the brain. Almost certainly communication and
> >> organizational systems in early living beings were reaction diffusion
> based.
> >> This is still a dominant effect for many 'sensing' in small organisms.
> >> Perhaps, therefore, one can look at nervous systems as structures
> >> specifically developed to supersede reaction diffusion mechanisms, thus
> >> superseding this very 'natural' but complexity limited type of
> communication
> >> and organization.  What this means, I believe, is that a simplified or
> >> abstracted physical or mathematical model of the brain explicitly
> violates
> >> the evolutionary pressures responsible for its structure.  Its where the
> >> wires go, what the wires do, and what the receiving neuron does with the
> >> information that forms the basis for neural computation, multiplied by a
> >> very large number.  And that is dependent on the actual physical
> structure
> >> of those elements.
> >>
> >> One more point about smart guys,  as a young computational
> neurobiologist I
> >> questioned how insightful John von Neumann actually was because I was
> >> constantly hearing about a lecture he wrote (but didn't give) at Yale
> >> suggesting that dendrites and neurons might be digital ( John von
> Neumann's
> >> The Computer and the Brain. (New Haven/London: Yale Univesity Press,
> 1958.)
> >> Very clearly a not very insightful idea for a supposedly smart guy.  It
> >> wasn't until a few years later, when I actually read the lecture - that
> I
> >> found out that he ends by stating that this idea is almost certainly
> wrong,
> >> given the likely nonlinearities in neuronal dendrites.  So von Neumann
> >> didn't lack insight, the people who quoted him did.  It is a remarkable
> fact
> >> that more than 60 years later, the majority of models of so called
> neurons
> >> built by engineers AND neurobiologists don't consider these
> nonlinearities.
> >> The point being the same point, to the Hopfield, Mead, Feynman list, we
> can
> >> now add Turing and von Neumann as suspecting that for understanding,
> >> biology and the nervous system must be dealt with in their full
> complexity.
> >>
> >> But thanks for the example from Turing - always nice to consider actual
> >> examples.   :-)
> >>
> >> Jim
> >>
> >>
> >>
> >>
> >>
> >> On Mar 19, 2014, at 8:30 PM, Geoffrey Goodhill <g.goodhill at uq.edu.au>
> wrote:
> >>
> >> Hi All,
> >>
> >> A great example of successful Ockham-inspired biology is Alan Turing's
> model
> >> for pattern formation (spots, stripes etc) in embryology (The chemical
> basis
> >> of morphogenesis, Phil Trans Roy Soc, 1953). Turing introduced a
> physical
> >> mechanism for how inhomogeneous spatial patterns can arise in a
> biological
> >> system from a spatially homogeneous starting point,  based on the
> diffusion
> >> of morphogens. The paper begins:
> >>
> >> "In this section a mathematical model of the growing embryo will be
> >> described. This model will be a simplification and an idealization, and
> >> consequently a falsification. It is to be hoped that the features
> retained
> >> for discussion are those of greatest importance in the present state of
> >> knowledge."
> >>
> >> The paper remained virtually uncited for its first 20 years following
> >> publication, but since then has amassed 8000 citations (Google
> Scholar). The
> >> subsequent discovery of huge quantities of molecular detail in
> biological
> >> pattern formation have only reinforced the importance of this relatively
> >> simple model, not because it explains every system, but because the
> >> overarching concepts it introduced have proved to be so fertile.
> >>
> >> Cheers,
> >>
> >> Geoff
> >>
> >>
> >> On Mar 20, 2014, at 6:27 AM, Michael Arbib wrote:
> >>
> >> Ignoring the gross differences in circuitry between hippocampus and
> >> cerebellum, etc., is not erring on the side of simplicity, it is erring,
> >> period. Have you actually looked at a Cajal/Sxentagothai-style drawing
> of
> >> their circuitry?
> >>
> >> At 01:07 PM 3/19/2014, Brian J Mingus wrote:
> >>
> >> Hi Jim,
> >>
> >> Focusing too much on the details is risky in and of itself. Optimal
> >> compression requires a balance, and we can't compute what that balance
> is
> >> (all models are wrong). One thing we can say for sure is that we should
> err
> >> on the side of simplicity, and adding detail to theories before simpler
> >> explanations have failed is not Ockham's heuristic. That said it's
> still in
> >> the space of a Big Data fuzzy science approach, where we throw as much
> data
> >> from as many levels of analysis as we can come up with into a big pot
> and
> >> then construct a theory. The thing to keep in mind is that when we start
> >> pruning this model most of the details are going to disappear, because
> >> almost all of them are irrelevant. Indeed, the size of the description
> that
> >> includes all the details is almost infinite, whereas the length of the
> >> description that explains almost all the variance is extremely short,
> >> especially in comparison. This is why Ockham's razor is a good
> heuristic. It
> >> helps prevent us from wasting time on unnecessary details by suggesting
> that
> >> we only inquire as to the details once our existing simpler theory has
> >> failed to work.
> >>
> >> On 3/14/14 3:40 PM, Michael Arbib wrote:
> >>
> >> At 11:17 AM 3/14/2014, Juyang Weng wrote:
> >>
> >> The brain uses a single architecture to do all brain functions we are
> aware
> >> of!  It uses the same architecture to do vision, audition, motor,
> reasoning,
> >> decision making, motivation (including pain avoidance and pleasure
> seeking,
> >> novelty seeking, higher emotion, etc.).
> >>
> >>
> >> Gosh -- and I thought cerebral cortex, hippocampus and cerebellum were
> very
> >> different from each other.
> >>
> >>
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140320/565bd255/attachment.html>