No subject
Tue Jun 6 06:52:25 EDT 2006
Here are some thoughts:
One general point is that I'm not entirely sure about what is
meant by theglobal/local distinction. Certainly action at a
distance can't take place; something physical happens to the
cell/connection in question in order for it to change. As I
understand it, the prototypical local learning is a
Hebbian rule, where all the information specifying plasticity
is in the pre and post-synaptic cells (ie "local" to the
connection), while a global learning rule is mediated by
something distal to the cell in question (i.e. a
neuromodulatory signal). But of course the signal must
contact the actual cell via diffusion of a chemical substance
(e.g. dopamine). So one different distinction might be how
specific the signal is; i.e. in a local rule like LTP the
information acts only on the single connection,
while a modulatory signal could change all the connections
in an area by a similar amount. However, the effects of a
neuromodulator could in turn be modulated by the current
state of the connection - hence a global signal might act
very differently at each connection. Which would make the
global signal seem local. So I'm not sure the distinction
is clearcut. Maybe its better to consider a continuum of
physical distance of the signal to change and specificity
of the signal at individual connections.
A couple of specific comments follow:
> A) Does plasticity imply local learning?
>
> The physical changes that are observed in synapses/cells in
> experimental neuroscience when some kind of external stimuli is
> applied to the cells may not result at all from any specific
> "learning" at the cells.The cells might simply be responding to a
> "signal to change" - that is, to change by a specific amount in a
> specific direction. In animal brains, it is possible that the
> "actual" learning occurs in some other part(s) of the brain, say
> perhaps by a global learning mechanism. This global mechanism can
> then send "change signals" to the various cells it is using to
> learn a specific task. So it is possible that in these
> neuroscience experiments, the external stimuli generates signals
> for change similar to those of a global learning agent in the
brain
> and that > the changes are not due to "learning" at the cells
> themselves. Please note that scientific facts/phenomenon like
LTP/LTD
> or synaptic plasticity can probably be explained equally well by
> many theories of learning (e.g. local learning vs. global
learning,
> etc.). However, the correctness of an explanation would have to
> be
I think it would be difficult to explain the actual phenomenon of
LTP/LTD as a response to some signal sent by a different part of
the brain, since a good amount of the evidence comes from in vitro
work. So clearly the "change signals" can't be coming from some
distant part of the brain - unless the slices contain the
necessary machinery for generating the change signal. Also, its of
course possible that LTP/LTD local learning rules act in concert
with global signals (as you mention below); these global signals
being sent by nonspecific neuromodulators (an idea brought up
plenty of times before). I'm not sure about the differences in the
LTP/LTD data collected in vivo versus in vitro; I'm sure there are
people out there studying it carefully, and this could provide
insight.
>
> B) "Pure" local learning does not explain a number of other
> activities that are part of the process of learning!!
>
> When learning is to take place by means of "local learning" in a
> network of cells, the network has to be designed prior to its
> training. Setting up the net before "local" learning can proceed
> implies that an external mechanism is involved in this part of
> the
> learning process. This "design" part of learning precedes actual
> training or learning by a collection of "local learners" whose
> only
> knowledge about anything is limited to the local learning law to
> use!
Of course, changing connection strengths seems to be the last phase
of the "learning/development" process. Correct numbers of cells
need
to be generated, they have to get to their correct locations,
proper
connections between subpopulations need to be established and
refined, and only at this point is there a substrate for "local"
learning. All of these can be affected to a certain extent by
environment. For example, the number of cells in the spinal cord
innervating a peripheral target can be downregulated with limb bud
ablation; conversely, the final number can be upregulated with
supernumerary limb grafts. Another well
known example is the development of ocular dominance columns. Here,
physical connections can be removed (in normal development), or new
connections can be established (evidence for this from the reverse
suture experiments), depending on the given environment. What would
be quite interesting would be if all these developmental phases are
guided by similar principles, but acting over different spatial and
temporal scales, and mediated by different carriers (e.g. chemical
versus electrical signals). Alas, if only I had a well-articulated,
cogent principle in hand with which to unify these disparate
findings; my first Nobel prize would be forthcoming. In lieu of
this, we're stuck with my ramblings.
>
> In order to learn properly and quickly, humans generally collect
> and store relevant information in their brains and then "think"
> about it (e.g. what problem features are relevant, problem
> complexity, etc.). So prior to any "local learning," there must
> be processes in the brain that examine this "body of
> information/facts" about a problem in order to design the
> appropriate network that would fit the problem complexity, select
> the problem features that are meaningful, etc. It would be very
> difficult to answer the questions "What size net?" and "What
> features to use?" without looking at the problem in great detail.
> A bunch of "pure" local learners, armed with their local learning
> laws, would have no clue to these issues of net design,
> generalization and feature selection.
>
> So, in the whole, there are a "number of activities" that need to
> be performed before any kind of "local learning" can take place.
> These aforementioned learning activities "cannot" be performed
> by a collection of "local learning" cells! There is more to the
> process of learning than simple local learning by individual
cells.
> Many learning "decisions/tasks" must precede actual training by
> "local learners." A group of independent "local learners" simply
> cannot start learning and be able to reproduce the learning
> characteristics and processes of an "autonomous system" like the
> brain.
>
> Local learning, however, is still a feasible idea, but only
> within a general global learning context. A global learning
> mechanism would be the one that "guides" and "exploits" these
> local learners. However, it is also possible that the global
> mechanism actually does all of the computations (learning)
> and "simply sends signals"
> to the network cells for appropriate synaptic adjustment. Both of
> these possibilities seem logical: (a) a "pure" global mechanism
> that learns by itself and then sends signals to the cells to
> adjust, or (b) a global/local combination where the global
> mechanism performs certain tasks and then uses the local
mechanism
> for training/learning.
>
> Note that the global learning mechanism may actually be
implemented
> with a collection of local learners!!
>
Notwithstanding the last remark, the above paragraphs perhaps run
the risk of positing a little global homunculus that "does all the
computations" and simply "sends signals" to the cells. I might be
confused by the distinction between local and global learning. All
we have to work with are cells that change their
properties based on signals impinging upon them, be they chemical
or electrical and originating near or far from the synapse, so it
seems that a "global" learning mechanism *must* be implemented by
local learners. (Again, if by local you specifically mean LTP/LTD
or something similar, then I agree - other mechanisms are also at
work).
> The basic argument being made here is that there are many tasks
> in a "learning process" and that a set of "local learners" armed
> with their local learning laws is incapable of performing all of
> those tasks. So local learning can only exist in the context of
> global learning and thus is only "a part" of the total learning
> process.
>
> It will be much easier to develop a consistent learning theory
> using the global/local idea. The global/local idea perhaps will
> also give us a better handle on the processes that we call
> "developmental" and "evolutionary."
One last comment. I'm not sure that the "developmental" vs.
"learning" distinction is meaningful, either (I'm not hacking on
your statements above, Asim; I think this distinction is more or
less a tacit assumption in pretty much all neuroscience research).
I read these as roughly equivalent to "nature vs. nurture" or
"genetics vs. environment". I would claim that to say that any
phenomenon is controlled by "genetics" is a scientifically
meaningless statement. The claim that such-and-such a phenomenon
is genetic is the modern equivalent of saying "The thing is there
cause thats how god made it". Genes don't code for behavioral
or physical attributes per se, they are simply a string of DNA
which code for different proteins. Phenotypes can only arise from
the genetic "code" by a complex interaction between cells and
signals from their environment. Now these signals can be generated
by events outside the organism or within the organism, and I would
say that the distinction between development and learning is better
thought of as whether the signals for change arise wholly within
the
organism or if the signals at least in part arise from outside the
organism. Any explanation of either learning or development has to
be couched in terms of what the relevant signals are and how they
affect the system in question.
anthony
============================================================
From: Russell Anderson, Ph.D.
Smith-Kettlewell Eye Research Institute
anderson at skivs.ski.org
I read over the replies you received with interest.
1. In regards to Response #1 (j. Faith)
I am not sure how relevant canalization is to your essay, but I
wrote a paper on the topic a few years back:
"Learning and Evolution: A Quantitative Genetics Approach"
J. Theor. Biol. 175:89-101 (1995).
Incidentally, the phenomenon known as "canalization" was described
much earlier by Baldwin, Osborn, and Morgan (in 1896), and is more
generally known as the "Baldwin effect" If you're interested, I
could mail you a copy.
2. I take issue with the analogies used by Brendan McCane.
His analogy of insect colonies is confused or irrelevant:
First, the behavior of insects, for the purpose of this argument,
does not indicate any individual (local) learning. Hence, the
analogy is inappropriate.
Second, The "global" learning occuring in the case of insect
colonies operates at the level of natural selection acting on the
genes, transmitted by the surviving colonies to new founding
Queens. In this sense, individual ants are genetically
ballistic ("pure developmental"). The genetics of insect colonies
are well-studied in evolutionary biology, and he should be referred
to any standard text on the topic (Dawkins, Dennett, Wilson, etc.)
The analogy using computer science metaphors is likewise flawed or
off-the-subject.
=============================================================
From: Steven M. Kemp |
Department of Psychology | email: steve_kemp at unc.edu
Davie Hall, CB# 3270 |
University of North Carolina |
Chapel Hill, NC 27599-3270 | fax: (919) 962-2537
I do not know if it is quite on point, but Larry Stein at the
University of California at Irvine has done fascinating work
on a very different type of
neural plasiticity called In-Vitro Reinforcement (IVR). I have
been working on neural networks whose learning algorithm is based
on his data and theory. I don't know whether you would call those
networks "local" or "global," but they do have the interesting
characteristic that all the units in the network receive the same
globally distributed binary reinforcement signal. That is,
feedback is not passed along the connections, but distributed
simultaneously and equally across the
network after the fashion of nondirected dopamine release from the
ventral tegmental projections.
In any event, I will forward the guts of a recent proposal we have
written here to give you a taste of the issues involved. I will be
happy to provide more information on this research if you are
interested.
(Steven Kemp did mail me parts of a recent proposal. It is long, so
I did not include it in this posting. Feel free to write to him or
me for a copy of it.)
============================================================
From: "K. Char" <kchar at elec.gla.ac.uk>
I have few quick comments:
1. The answer to some parts of the discussions seem to lie in the
notion of a *SEQUENCE*. That is: global->local->(final) global;
clearly the initial global is not the same as the final global.
Some of the discussants seem to prefer the sequence: local->global.
A number of such possibilities exists.
2. The next question is: who dictates the sequence? Is it a global
mechanism or a local mechanism?
3. In the case of the bee, though it had an individual goal how
was this goal arrived at?
4. In the context of neural networks (artificial or real): who
dictates the node activation functions, the topology and the
learning rules? Does every node find its own activation function?
5. Finally how do we form concepts? Do the concepts evolve as a
result of local interactions at the neuron level or through the
interaction of micro-concepts at a global level which then trigger
a local mechanism?
6. Here the next question could be: how did these micro-concepts
evolve in the very first place?
7. Is it possible that these neural structures provide the *very
motivation* for the formation of concepts at the global level in
order to adapt these structures effectively? If so, does this
motivation arise from the environment itself?
============================================================
Response # 1:
As you mention, neuroscience tends to equate network plasticity
with learning. Connectionists tend to do the same. However this
raises a problem with biological systems because this conflates the
processes of development and learning. Even the smartest organism
starts from an egg, and develops for its entire lifespan - how do
we distinguish which changes are learnt, and which are due to
development. No one would argue that we *learn* to have a cortex,
for instance, even though it is due to massive emryological changes
in the central nervous system of the animal.
This isn't a problem with artificial nets, because they do not
usually have a true developmental process and so there can be no
confusion between the two; but it has been a long-standing problem
in the ethology literature, where learnt changes are contrasted
with "innate" developmental ones. A very interesting recent
contribution to this debate is Andre Ariew's "Innateness and
Canalization", in Philosophy of Science 63 (Proceedings), in which
he identifies non-learnt changes as being due to canalised
processes. Canalization was a concept developed by
the biologist Waddington in the 40's to describe how many changes
seem to have fixed end-goals that are robust against changes in
the environment.
The relationship between development and learning was also
thoroughly explored by Vygotsky (see collected works vol 1, pages
194-210).
I'd like to see what other sorts of responses you get,
Joe Faith <josephf at cogs.susx.ac.uk>
Evolutionary and Adaptive Systems Group,
School of Cognitive and Computing Sciences,
University of Sussex, UK.
=================================================================
Response # 2:
I fully agree with you, that local learning is not the one and only
ultimate approach - even though it results in very good learning
for some domains.
I am currently writing a paper on the competitive learning
paradigm. I am proposing, that this competition that occurs e.g.
within neurons should be called local competition. The network as a
whole gives a global common goal to these local competitors and
thus their competition must be regarded as cooperation from a more
global point of view.
There is a nice paper by Kenton Lynne that integrates the ideas of
reinforcement and competition. When external evaluations are
present, they can serve as teaching values, if nor the neurons
compete locally.
@InProceedings{Lynne88,
author = {K.J.\ Lynne},
title = {Competitive Reinforcement Learning},
booktitle = {Proceedings of the 5th International Conference
on Machine Learning},
year = {1988},
publisher = {Morgan Kaufmann},
pages = {188--199}
}
----------------------------------------------------------
Christoph Herrmann Visiting researcher
Hokkaido University
Meme Media Laboratory
Kita 13 Nishi 8, Kita- Tel: +81 - 11 - 706 - 7253
Sapporo 060 Fax: +81 - 11 - 706 - 7808
Japan Email: chris at meme.hokudai.ac.jp
http://aida.intellektik.informatik.th-darmstadt.de/~chris/
=============================================================
Response #3:
I've just read your list of questions on local vs. global learning
mechanisms. I think I'm sympathatic to the implications or
presuppositions of your questions but need to read them more
carefully later. Meanwhile, you might find very interesting a
two-part article on such a mechanism by Peter G. Burton in the 1990
volume of _Psychobiology_ 18(2).119-161 & 162-194.
Steve Chandler
<chandler at uidaho.edu>
===============================================================
Response #4:
A few years back, I wrote a review article on issues of local
versus global learning w.r.t. synaptic plasticity. (Unfortunately,
it has been "in press" for nearly 4 years). Below is an abstract. I
can email the paper to you in TeX or
postscript format, or mail you a copy, if you're interested.
Russell Anderson
------------------------------------------------
"Biased Random-Walk Learning:
A Neurobiological Correlate to Trial-and-Error"
(In press: Progress in Neural Networks)
Russell W. Anderson
Smith-Kettlewell Eye Research Institute
2232 Webster Street
San Francisco, CA 94115
Office: (415) 561-1715
FAX: (415) 561-1610
anderson at skivs.ski.org
Abstract:
Neural network models offer a theoretical testbed for the study of
learning at the cellular level. The only experimentally verified
learning rule, Hebb's rule, is extremely limited in its ability to
train networks to perform complex tasks.
An identified cellular mechanism responsible for Hebbian-type
long-term potentiation, the NMDA receptor, is highly versatile.
Its function and efficacy are modulated by a wide variety of
compounds and conditions and are likely to be directed by non-local
phenomena. Furthermore, it has been demonstrated that NMDA
receptors are not essential for some types of learning. We have
shown that another neural network learning rule, the chemotaxis
algorithm, is theoretically much more powerful than Hebb's rule and
is consistent with experimental data. A biased random-walk in
synaptic weight space is a learning rule immanent in nervous
activity and may account for some types of learning -- notably the
acquisition of skilled movement.
==========================================================
Response #5:
Asim Roy typed ...
>
> B) "Pure" local learning does not explain a number of other
> activities that are part of the process of learning!!
..
>
> So, in the whole, there are a "number of activities" that need to
> be
> performed before any kind of "local learning" can take place.
> These aforementioned learning activities "cannot" be performed by
> a collection of "local learning" cells! There is more to the
> process of learning than simple local learning by individual
cells.
> Many learning "decisions/tasks" must precede actual training by
> "local learners." A group of independent "local learners" simply
> cannot start learning and be able to reproduce the learning
> characteristics and processes of an "autonomous system" like the
> brain.
I cannot see how you can prove the above statement (particularly
the last sentence). Do you have any proof. By analogy, consider
many insect colonies (bees, ants etc). No-one could claim that one
of the insects has a global view of what should happen in the
colony. Each insect has its own purpose and goes about that purpose
without knowing the global purpose of the colony. Yet an ants nest
does get built, and the colony does survive. Similarly, it is
difficult to claim that evolution has a master plan, order just
seems to develop out of chaos.
I am not claiming that one type of learning (local or global) is
better than another, but I would like to see some evidence for your
somewhat outrageous claims.
> Note that the global learning mechanism may actually be
implemented
> with a collection of local learners!!
You seem to contradict yourself here. You first say that local
learning cannot cope with many problems of learning, yet global
learning can. You then say that global learning can be implemented
using local learners. This is like saying that you can implement
things in C, that cannot be implemented in assembly!! It may be
more convenient to implement it in C (or using global learning),
but that doesn't make it impossible for assembly.
-------------------------------------------------------------------
Brendan McCane, PhD. Email:
mccane at cs.otago.ac.nz
Comp.Sci. Dept., Otago University, Phone: +64 3 479 8588.
Box 56, Dunedin, New Zealand. There's only one catch -
Catch 22.
===============================================================
Response #6:
In regards to arguments against global learning:I think no one
seriously questions this possibility, but think that global
learning theories are currently
non-verifiable/ non-falsifyable. Part of the point of my paper was
that there ARE ways to investigate non-local learning, but it
requires changes in current experimental protocols.
Anyway, good luck. I look forward to seeing your compilation.
Russell Anderson
2415 College Ave. #33
Berkeley, CA 94704
==============================================================
Response #7:
I am sorry that it has taken so long for me to reply to
your inquiry about plasticity and local/global learning. As I
mentioned in my first note to you, I am sympathetic to the view
that learning involves some sort of overarching, global mechanism
even though the actual information storage may consist of
distributed patterns of local information. Because I am
sympathetic to such a view, it makes it very difficult for me to
try to imagine and anticipate the problems
for such views. That's why I am glad to see that you are
explicitly trying to find people to point out possible problems; we
need the reality check.
The Peter Burton articles that I have sent you describes
exactly the kind of mechanism implied by your first question: Does
plasticity imply local learning? Burton describes a neurological
mechanism by which local learning could emerge from a global
signal. Essentially he posits that whenever the new perceptual
input being attended to at any given moment differs sufficiently
from the record of previously recorded experiences to which that
new input is being compared, the difference triggers a global
"proceed-to-store" signal. This signal creates a neural
"snapshot" (my term, not Burton's) of the cortical activations at
that moment, a global episodic memory
(subject to stimulus sampling effects, etc.). Burton goes on to
describe how discrete episodic memories could become associated
with one another so as to give rise to schematic representations of
percepts (personally I don't think that positing this abstraction
step is necessary, but Burton does it).
As neuroscientists sometimes note, while it is widely
assumed that LTP/LTD are local learning mechanisms, the direct
evidence for such a hypothesis is pretty slim at best. Of course
of of the most serious problems with that view is that the changes
don't last very long and thus are not really good candidates for
long term (i.e., life long) memory. Now, to my mind, one of the
most important possibilities overlooked in LTP studies
(inherently so in all in vitro preparations and so far as I know
--which is not very far because this is not my
field--in the in vivo preparations that I have read about) is that
LTP/D is either an artifact of the experiment or some sort of short
term change which requires a global signal to become consolidated
into a long term record. Burton describes one such possible
mechanism.
Another motivation for some sort of global mechanism comes
from the so-called 'binding problem' addressed especially by the
Damasio's, but others too. Somehow somewhere all the distributed
pieces of information about what an orange is, for example, have to
be tied together. A number of studies of different sorts have
demonstarted repeatedly that such information is distributed
throughout cortical areas.
Burton distinguishes between "perceptual learning"
requiring no external teacher (either locally or globally) and
"conceptual learning", which may require the assistance of a
'teacher'. In his model though, both types of learning are
activated by global "proceed-to-learn" signals triggered in turn by
the global summation of local disparities between remembered
episodes and current input.
I'll just mention in closing that I am particularly
interested in the empirical adequacy of neuropsychological accounts
such as Burton's because I am very interested in "instance-based"
or "exemplar-based" models of learning. In particular, Royal
Skousen's _Analogical Modeling of Language_ (Kluwer, 1989)
describes an explicit, mathematical model for predicting new
behavior on analogy to instances stored in long term memory.
Burton's model suggests a possible neurological basis for such
behavior.
Steve Chandler
<chandler at uidaho.edu>
==============================================================
Response #8:
*******************************************************************
Fred Wolf E-Mail:
fred at chaos.uni-frankfurt.de
Institut fuer Theor. Physik
Robert-Mayer-Str. 8 Tel: 069/798-23674
D-60 054 Frankfurt/Main 11 Fax: (49) 69/798-28354
Germany
could you please point me to a few neuroBIOLOGICAL references that
justify your claim that
>
> A predominant belief in neuroscience is that synaptic plasticity
> and LTP/LTD imply local learning (in your sens).
>
I think many people appreciate that real learning implies the
concerted interplay of a lot of different brain systems and should
not even be attempted to be explained by "isolated local learners".
See e.g. the series of review-papers on memory in a recent volume
of PNAS 93 (1996) (http://www.pnas.org/).
Good luck with your general theory of global/local learning.
best wishes
Fred Wolf
==============================================================
Response #9:
I am into neurocomputing for several years. I read your arguments
with interest. They certainly deserve further attention. Perhaps
some combination of global-local learning agents would be the right
choice.
- Vassilis G. Kaburlasos
Aristotle University of Thessaloniki, Greece
==============================================================
===============================================================
Original Memo:
A predominant belief in neuroscience is that synaptic plasticity
and LTP/LTD imply local learning. It is a possibility, but it is
not the only possibility. Here are some thoughts on some of the
other possibilities (e.g. global learning mechanisms or a
combination of global/local mechanisms) and some discussion on the
problems associated with "pure" local learning.
The local learning idea is a very core idea that drives research in
a number of different fields. I welcome comments on the questions
and issues raised here.
This note is being sent to many listserves. I will collect all of
the responses from different sources and redistribute them to all
of the participating listserves. The last such discussion was very
productive. It has led to the realization by some key researchers
in the connectionist area that "memoryless" learning perhaps is not
a very "valid" idea. That recognition by itself will lead to more
robust and reliable learning algorithms in the future. Perhaps a
more active debate on the local learning issue will help us resolve
this issue too.
A) Does plasticity imply local learning?
The physical changes that are observed in synapses/cells in
experimental neuroscience when some kind of external stimuli is
applied to the cells may not result at all from any specific
"learning" at the cells. The cells might simply be responding to a
"signal to change" - that is, to change by a specific amount in a
specific direction. In animal brains, it is possible that the
"actual" learning occurs in some other part(s) of the brain, say
perhaps by a global learning mechanism. This global mechanism can
then send "change signals" to the various cells it is using to
learn a specific task. So it is possible that in these neuroscience
experiments, the external stimuli generates signals for change
similar to those of a global learning agent in the brain and that
the changes are not due to "learning" at the cells themselves.
Please note that scientific facts and phenomenon like LTP/LTD or
synaptic plasticity can probably be explained equally well by many
theories of learning (e.g. local learning vs. global learning,
etc.). However, the correctness of an explanation would have to be
judged from its consistency with other behavioral and biological
facts, not just "one single" biological phenomemon or fact.
B) "Pure" local learning does not explain a number of other
"activities" that are part of the process of learning!!
When learning is to take place by means of "local learning" in a
network of cells, the network has to be designed prior to its
training. Setting up the net before "local" learning can proceed
implies that an external mechanism is involved in this part of the
learning process. This "design" part of learning precedes actual
training or learning by a collection of "local learners" whose only
knowledge about anything is limited to the local learning law to
use! In addition, these "local learners" may have to be told what
type of local learning law to use, given that a variety of
different types can be used under different circumstances. Imagine
who is to "instruct and set up" such local learners which type of
learning law to use? In addition to these, the "passing" of
appropriate information to the appropriate set of cells also has
to be "coordinated" by some external or global learning mechanism.
This coordination cannot just happen by itself, like magic. It has
to be directed from some place by some agent or mechanism.
In order to learn properly and quickly, humans generally collect
and store relevant information in their brains and then "think"
about it (e.g. what problem features are relevant, complexity of
the problem, etc.). So prior to any "local learning," there must be
processes in the brain that "examine" this "body of
information/facts" about a problem in order to design the
appropriate network that would fit the problem complexity, select
the problem features that are meaningful, etc. It would be very
difficult to answer the questions "What size net?" and "What
features to use?" without looking at the problem (body of
information)in great detail. A bunch of "pure" local learners,
armed with their local learning laws, would have no clue to these
issues of net design, generalization and feature selection.
So, in the whole, there are a "number of activities" that need to
be performed before any kind of "local learning" can take place.
These aforementioned learning activities "cannot" be performed by a
collection of "local learning" cells! There is more to the process
of learning than simple local learning by individual cells. Many
learning "decisions/tasks" must precede actual training by "local
learners." A group of independent "local learners" simply cannot
start learning and be able to reproduce the learning
characteristics and processes of an "autonomous system" like the
brain.
Local learning or local computation, however, is still a feasible
idea, but only within a general global learning context. A global
learning mechanism would be the one that "guides" and "exploits"
these local learners or computational elements. However, it is also
possible that the global mechanism actually does all of the
computations (learning) and "simply sends signals" to the network
of cells for appropriate synaptic adjustment. Both of these
possibilities seem logical: (a)
a "pure" global mechanism that learns by itself and then sends
signals to the cells to adjust, or (b) a global/local combination
where the global mechanism performs certain tasks and then uses the
local mechanism for training/learning.
Thus note that the global learning mechanism may actually be
implemented with a collection of local learners or computational
elements!! However, certain "learning decisions" are made in the
global sense and not by "pure" local learners.
The basic argument being made here is that there are many tasks in
a "learning process" and that a set of "local learners" armed with
their local learning laws is incapable of performing all of those
tasks. So local learning can only exist in the context of global
learning and thus is only "a part" of the total learning process.
It will be much easier to develop a consistent learning theory
using the global/local idea. The global/local idea perhaps will
also give us a better handle on the processes that we call
"developmental" and "evolutionary." And it will, perhaps, allow us
to better explain many of the puzzles and inconsistencies in our
current body of discoveries about the brain. And, not the least, it
will help us construct far better algorithms by removing the
"unwarranted restrictions" imposed on us by the current ideas. Any
comments on these ideas and possibilities are welcome.
Asim Roy
Arizona State University
More information about the Connectionists
mailing list