Network Constructing Algorithms.

[Tal Grossman]

Network constructing algorithms, i.e. learning algorithms which add units
while training, receive a lot of interest these days. I've recently
compiled a reference list of papers presenting such algorithms. I send
this list as a small contribution to the last discussion. I hope people
will find it relevant and usefull.  Of course, it is probably not
exhaostive - and I'd like to hear about any other related work.
Note that two refs. are quite old (Hopcroft and Cameron) - from the threshold
logic days. A few papers include convergence proofs (Frean, Gallant,
Mezard and Nadal, Marchand et al). Naturally, there is a significant
overlap between some of the algorithms/architecture.

I also appologize for the primitive Tex format.

                                               Tal grossman < fegross at weizmann>
                                               Electronics Dept.
                                               Weizmann Inst.
                                               Rehovot 76100, ISRAEL.
-------------------------------------------------------------------------------

\centerline{\bf Network Generating Learning Algortihms - Refernces.}

T. Ash, ``Dynamic Node Creation in Back-Propagation Networks",
Tech.Rep.8901, Inst. for Cognitive Sci., Univ. of California, San-Diego.

Cameron S.H., ``The Generation of Minimal Threshold Nets by an
Integer Program",
IEEE TEC {\bf EC-13},299 (1964).

S.E. Fahlman and C.L. Lebiere,
``The Cascade-Correlation Learning Architecture",
in {\it Advances in Neural Information Processing Systems 2},
D.S. Touretzky ed. (Morgan Kaufmann, San Mateo 1990), pp. 524.

M. Frean, ``The Upstart Algorithm: a Method for Constructing and
Trainig Feed Forward Neural Networks",
Neural Computation {\bf 2}:2 (1990).

S.I. Gallant, ``Perceptron -Based Learning Algorithms", IEEE Trans. on
Neural Networks {\bf 1}, 179 (1990).

M. Golea and M. Marchand, ``A Growth Algorithm for Neural Network
Decision Trees", EuroPhys.Lett. {\bf 12}, 205 (1990).

S.J. Hanson, ``Meiosis Networks",
in {\it Advances in Neural Information Processing Systems 2},
D.S. Touretzky ed. (Morgan Kaufmann, San Mateo 1990), pp. 533.

Honavar V. and Uhr L.
in the {\it Proc. of the 1988 Connectionist
Models Summer School}, Touretzky D., Hinton G. and Sejnowski T. eds.
(Morgan Kaufmann, San Mateo, 1988).

Hopcroft J.E. and Mattson R.L.,
``Synthesis of Minimal Threshold Logic Networks",
IEEE TEC {\bf EC-14}, 552 (1965).

Mezard M. and Nadal J.P.,
``Learning in Feed Forward Layered Networks - The Tiling Algorithm",
J.Phys.A {\bf 22}, 2129 (1989).

J.Moody, ``Fast Learning in Multi Resolution Hierarchies",
in {\it Advances in Neural Information Processing Systems 1},
D.S. Touretzky ed. (Morgan Kaufmann, San Mateo 1989).

J.P. Nadal, ``Study of a Growth Algorithm for a Feed Forward Network",
International J. of Neural Systems {\bf 1}, 55 (1989).

Rujan P. and Marchand M.,
``Learning by Activating Neurons: A New Approach to Learning in
Neural Networks",
Complex Systems {\bf 3}, 229 (1989); and also in the
{\it Proc. of the First International
Joint Conference on Neural Networks -
Washington D.C. 1989}, Vol.II, pp.105.

J.A. Sirat and J.P. Nadal, ``Neural Trees: A New Tool for Classification",
preprint, submitted to "Network", April 90.
\bye