network labeling conventions

[john moody]

I agree with Scott Fahlman that there is need for a standard
labeling convention for multilayered networks. The convention
which I prefer for an "N Layer Network" is diagramed below. 
Such a network has "(N-1) Internal Layers of Units" and "N
Layers of Weights".  Each connection has the same layer index 
as its post-synaptic processing unit. The output units are "Layer N". 
The input lines are not enumerated as a layer since they usually do 
no processing; for consistency, however, the input lines can be 
identified as "Layer 0".  As a matter of style, I think it is 
confusing to use the same graphic symbols for input lines as 
for the non-linear processing units, since any operation performed 
on the input data prior to its arrival at the first layer of 
connections is really pre-processing and not part of the network 
computation proper.  Along the same lines, it would be useful to 
use a distinguishing symbol when linear output units are used in 
networks which perform mappings from R^n to R^m.

Layer N Units (Outputs) O  O  O  O  O  O    Activations  A^N_n
Layer N Weights              /|\            Weight Values  W^N_nm
                            / | \
Layer N-1 Units         O  O  O  O  O  O    Activations  A^(N-1)_m
Layer N-1 Weights            /|\            Weight Values  W^(N-1)_ml
                            / | \
      .                       .
      .                       .

Layer 2 Units           O  O  O  O  O  O    Activations  A^2_k
Layer 2 Weights              /|\            Weight Values  W^2_kj
                            / | \
Layer 1 Units           O  O  O  O  O  O    Activations  A^1_j
Layer 1 Weights              /|\            Weight Values  W^1_ji
                            / | \
Layer 0 (Input Lines)   .  .  .  .  .  .    Input Activations  A^0_i


--John Moody
-------