Connectionists: AI at 50 videos...

[Barak A. Pearlmutter]

Dear Danny,

If I might paraphrase that work, it seems to me the idea is that
"subsymbolic" signals are used for local processing, while symbolic
representations are used for longer distance communication, with
things becoming more symbolic the longer and narrower the channel:
from one brain region to another, from the brain to its future self,
from one brain to another.

That certainly seems consistent with the information bottleneck of
Tishby et al, where distributions fragment into clustered
representations as the bottleneck becomes more severe.

My only issue is that, if you measure things in a conventional
computer doing symbol processing, the individual bits on wires can be
seen as subsymbolic. What is bit 7 of register 4? What is bit 11 of
the address bus? It is not until you agglomerate things just right, at
just the right level of abstraction, that the crisp symbolic nature
of, say, a compiler running on a simple RISC, becomes apparent. This
subproblem seems to subsume the whole question, because it's basically
"figure out how it all works and then we can talk about whether there
are symbolic representations." This might not obviate the distinction
between symbolic and subsymbolic, but I'd argue that it does make it
uninteresting.

Cheers,

--Barak.