No subject

appropriate to distinguish between "memory" and "learning".
Memory is simple recording of information and facts (word
pairs, etc), whereas learning involves generalization
(deriving additional general facts from that information).
Generalization (e.g. categorization) depends on a body of
information. Recording of names of people, objects, places
and relations is simple memorization, not learning
(generalization). Much of what is cited by Gale Martin,
Gary Cottrell and Stefan Schaal is recording of information
(memorization).

And "relevant" information, facts may indeed be recorded
"instantaneously" (memorized) by the brain, there is no
question about that. The issue here is whether "learning"
is instantaneous and permanent in the Hebbian sense. The
study clearly indicates that is not so.

-------------------
(C) COULD THE "LOSS OF SKILL" RESULT FROM TRAINING OF THE 
SAME NET, HEBBIAN STYLE (THE INTERFERENCE PROBLEM)?

Several people raised this question (Eric Pitcher, Will 
Penny and others). There are a variety of problems with 
that argument. First, the same network may not be
appropriate for learning the second motor skill. So, in 
that case, there are two possibilities to consider for any
learning system, biological or otherwise. One, it could
destroy the previous net and use its free neurons to create
a new net (perhaps using more or less neurons than the
previous net) to learn the second motor skill. But such
distruction of the previous net will result in "total" loss

of skill on the first task, not just "partial" loss of
skill. So that possibility will not explain the phenomenon
at hand.

Second, if in fact the same net is used to learn the second 
motor skill, then one would have the problem of
"catastrophic forgetting." As is well-known, catastrophic
forgetting is observed in back-propagation and other types
of networks when a previously trained network is
subsequently trained with examples from a problem that is
completely different from the previous one. And 
catastrophic forgetting is not just limited to pathological
cases of learning. Catastrophic forgetting, in fact, is
what we "depend on" when we talk about "adaptive learning"
- adaptating to a new situation and forgetting the old. So
learning of the second skill in the same net would also
result in "total loss of skills" (catastrophic forgetting),
not just "partial loss of skills." So this does not explain
the phenomenon either.

So this type of interference is not a good explanation for
the phenomenon at hand - that of "partial loss of skills."

--------------------
D) WHY DO YOU SAY CLASSICAL CONNECTIONIST LEARNING IS 
MEMORYLESS? ISN'T THERE MEMORY IN THE WEIGHTS?

Several persons raised this issue. So I include this note
below from one of my previous memos:

"Memoryless learning implies there is no EXPLICIT storage 
of any learning example in the system in order to learn. In
classical connectionist learning, the weights of the net 
are adjusted whenever a learning example is presented, but
it is promptly forgotten by the system. There is no 
EXPLICIT storage of any presented example in the system.
That is the generally accepted view of "adaptive" or
"on-line learning systems."

Imagine such a system "planted" in some human brain. And
suppose we want to train it to learn addition. So we 
provide the first example - say, 2 + 2 = 4. This system 
then uses the example to promptly adjust the weights of the
net and forgets the particular example. It has done what it
is supposed to do - adjust the weights, given a learning
example. Suppose, you then ask this "human", fitted with
this learning algorithm: "How much is 2 + 2?" Since it has
only seen one example and has not yet fully grasped the 
rule for adding numbers, it probably would give a wrong
answer. So you, as the teacher, perhaps might ask at that
point: "I just told you 2 + 2 = 4. What do you mean you
don't remember?" And this "human" might respond: "Very
honestly, I don't recall you ever having said that!
I am very sorry." And this would continue to happen after
every example you present to this "human" until complete
learning has taken place!!!

So do you think there is memory in those "weights"? Do you
think humans are like that?"

------------------------
(E) A LAST NOTE: The arguments I used against Hebbian-style
learning did not rely in any way or form on the details of
the PET studies. Only the external behavioral facts were

used in the arguments. So questions about irreproducibility
of PET and fMRI studies are irrelevant to this argument.

-------------------------------------------
RESPONSES FROM OTHERS