distributed/local

[Nigel.Goddard@B.GP.CS.CMU.EDU]

Both extremes are wrong for representing conceptual knowledge (i.e.,
one unit per concept versus all units participate in all concepts).
Disadvantages of extreme local include no tolerance to failure (neurons
die all the time), difficult to express nuance without impossibly large
numbers of units.  The big advantage is it is easy to see what is going
on, to design structures.  Disadvantages of extreme distributed include
crosstalk when more than one item is active and difficulty communicating
an active item from one part of the architecture to another (too many links
required).  The big advantages are fault-tolerance (graceful degradation)
and generalization.  The answer is something in between the extremes
(not that this is news to anyone), depending on what the task is.  Order
logn units per concept for an n-unit net might be a good place to start.  

Feldman has a TR discussing these issues in much more depth (TR 189,
"Neural Representation of Conceptual Knowledge", Computer Science Dpt,
Univ. Rochester, NY 14627).  Also published as a book chapter, I believe.


Nigel Goddard