Structure of Neural Nets - control

[tap@nmsu.csnet]

> Modular structure is interesting so that it expands the
> net's heterogeneous structure while maintaining rather
> uniform structure in a layer within a module. Each module
> may be treated and controlled as a unit at another level.
> - Teru Homma

This raises a whole lot of questions:

Do we really want strict modular structure, where each
module is controlled as a unit at another (higher) level?
-- Some structural modularity may be neccessary, but maybe
the type of modularity that works best for dealing with the
world is a flexible one, where the boundaries and interfaces
of modules are flexible and context-dependent.

How are these modules at that higher level controlled? -
as units at an even higher level?

Where does it end?, -- it must end in some
self-regulated unit or module.

Will this result in the type of inflexible control that
one sees in many discrete-symbol AI systems?

Can we develop principles of self-regulation and apply
them directly to all modules, rather than having a hierarchy
of control?  Is Grossberg's ART a step in this direction?

Nearly all work by connectionists that is related by AI
has been on developing better representations for data.
Most of the processing has been of a single step (single
forward propagation, single relaxation, or single settling),
and consequently the control has been very simple.  Very
little work has been done on problems that require more
complex control of processing, such as planning, or
analysing sequential input of unbounded length and
complexity.  The control structures used in discrete-symbol
AI for doing this type of processing have the same
inflexibilities as the discrete-symbol representations.  Can
connectionists do for control structures what they are doing
for representations? i.e. decompose them and recompose them
in a more flexible and accessible way.  Can representation
of control and representation of data become the same thing?
I mean this in a strong sense, I don't mean that they should
just be representable using the same techniques, but rather
that they be identical: the representation of data
represents by virtue of recording (memory) and creating
(recall, action) processes that that data has effect upon,
(and there is NOTHING else).

Is there any real and/or valuable distinction between
process and control, or is it just a convenient way of
interpreting complex systems? (and what is process and what
is control?).  I would claim that discrete-symbol AI has
often made this distinction.  It is most evident in systems
which systems which provide automatic-backtracking the
control decisions (which goal to select next, how to
backtrack) are usually made by the system and not the
program which is running in it (e.g. most Prologs).  I also
claim this is bad: systems whose designs contain explicit or
implicit distinctions between control and process are doomed
to inflexible hierarchical control.  But can it be
otherwise?  -- yes -- some prologs (e.g. Nu-Prolog) allow
the data to affect control decisions - goal selection is
dependent upon patterns of instantiation. I think this is a
step in the right direction, and one that can be taken much
further by connectionism.

-- Tony Plate