Similarity to Cascade-Correlation

[Tom Dietterich]

As someone with a lot of experience in decision-tree learning
algorithms, I agree with Scott.  The main similarity between
Cascade-Correlation (CC) and decision tree algorithms like CART is
that they are both greedy.  CART and related algorithms (e.g., ID3,
C4, CN2, GREEDY3) all work by choosing an (axis-parallel) hyperplane
and then subdividing the training data along that hyperplane, whereas
CC keeps all of the training data together and keeps retraining the
output units as it incrementlly adds hidden units.

There is an algorithm, called FRINGE, that learns a decision tree and
then uses that tree to define new features which are then used to
build a new tree (and this process can be repeated, of course).  This
is the best example I know of a non-connectionist (supervised)
algorithm for defining new features.

--Tom