Connectionist Learning - Some New Ideas

[Steven Small]

>Dr. Roy's suggestion that the brain must try "to design the smallest possible
>net for every task" because "the brain could not be wasteful of its limited
>resources" is unlikely, in my opinion. It seems to me that the brain has
>rather an abundance of neurons. On the other hand, finding optimal solutions to
>many interesting "real-world" problems is often very hard computationally. I am
>not a complexity theorist, but I will hazard to suggest that a constraint on
>neural systems to be optimal or near-optimal in their space usage is probably
>both impossible to realize and, in fact, unnecessary.
>
>Wild speculation: the brain may have so many neurons precisely so that it can
>afford to be suboptimal in its storage usage in order to avoid computational
>time intractability.

I agree with this general idea, although I'm not sure that "computational
time intractability" is necessarily the principal reason. There are a lot
of good reasons for redundancy, overlap, and space "suboptimality", not the
least of which is the marvellous ability at recovery that the brain
manifests after both small injuries and larger ones that give pause even to
experienced neurologists.

-SLS