On the Care & Feeding of Learning Models

[Stevan Harnad]

Tom Bever (bevr at db1.cc.rochester.edu) wrote:

>   Recent correspondence has focussed on the performance level of the
>   Rumelhart and McClelland past tense learning model and subsequent
>   models, under varying conditions of feeding... We have demonstrated
>   (Lachter and Bever, 1988; Bever, 1988) that existing connectionist
>   models learn to simulate rule-governed behavior only insofar as the
>   relevant structures are built into the model or the way it is fed.
>   What would be important to show is that such models could achieve
>   the same performance level and characteristics without structures
>   and feeding schemes which already embody what is to be learned.

I don't understand the "feeding" metaphor. If I feed an inductive
device data that have certain regularities, along with feedback as to
what the appropriate response would be (say, for the sake of
simplicity, they are all members of a dichotomy: Category C or
Category Not-C), and the device learns to perform the response (here,
dichotomization), presumably by inducing the regularities
statistically from the data, what is there about this "feeding"
regimen that "already embodies what is to be learned" (and hence,
presumably, constitutes some sort of cheating)? Rather than cheating,
it seems to me that rules that are come by in this way are the wages
of "honest toil."

Perhaps there is a suppressed "poverty-of-the-stimulus" premise here,
to the effect that we are only considering data that are so
underdetermined as to make their underlying regularities uninducible
(i.e., the data do not sufficiently "embody" their underlying
regularities to allow them to be picked up statistically). If this is
what Bever has in mind, it would seem that this putative poverty has to
be argued for explicitly, on a case by case basis.

Or is the problem doubts about whether nets can do nontrivial
generalization from their data-sets? But then wouldn't this too have
to be argued separately?

But objections to "feeding conditions" alone...? Is the objection that
nets are being spoon-fed, somehow? How? Trial-and-error-sampling sounds
more like doing it the old-fashioned way. Biased samples? Loaded samples?

Stevan Harnad