BP for categorization...

[Yves (Zip) Lacouture]

> From: xiru at com.think
> Subject: backprop for classification
> Date: 19 Aug 90 00:26:28 GMT
>
> While we trained a standard backprop network for some classification task
> (one output unit for each class), we found that when the classes are not
> evenly distribed in the training set, e.g., 50% of the training data belong
> to one class, 10% belong to another, ... etc., then the network always biased
> towards the classes that have the higher percentage in the training set.
>

I encountered the same problem in a similar situation. This occur with
limited resources (HU): the network tend to neglet a subset of the
stimuli. The phenomenon is also observed when the stimuli have the same
presentation probability and the resources are very limited. It helps to
use a non-orthogonal representation (e.g. by activating neighbor units).
To build a model of (human) simple identification I modified BP to
incorporate a selective attention mechanism by which the adaptative
modifications are made larger for the stimuli for which performances are
worse. I expect to offer a TR on this topic soon.

yves