Tech. Report on Unsupervised Learning

[Joachim M. Buhmann]

The following technical report on Unsupervised Learning is now 
available from our website

  http://www-dbv.informatik.uni-bonn.de/papers.html#NeuralNetworks

                  Empirical Risk Approximation: 
        An Induction Principle for Unsupervised Learning

                        Joachim M. Buhmann
                Institute for Computer Science (III), 
                        University of Bonn


Unsupervised learning algorithms are designed to extract structure
from data without reference to explicit teacher information. The
quality of the learned structure is determined by a cost function
which guides the learning process. This paper proposes Empirical Risk
Approximation as a new induction principle for unsupervised learning.
The complexity of the unsupervised learning models are automatically
controlled by the two conditions for learning: (i) the empirical risk
of learning should uniformly converge towards the expected risk; (ii)
the hypothesis class should retain a minimal variety for consistent
inference. The maximal entropy principle with deterministic annealing
as an efficient search strategy arises from the Empirical Risk
Approximation principle as the optimal inference strategy for large
learning problems. Parameter selection of learnable data structures is
demonstrated for the case of K-means clustering.
---------------------------------------------------------------------
Joachim M. Buhmann
Institut fuer Informatik III	       Tel.(office) : +49 228 734 380
Universitaet Bonn		       Tel.(secret.): +49 228 734 292
Roemerstr. 164					Fax:  +49 228 734 382
D-53117 Bonn			     email: jb at informatik.uni-bonn.de
Fed. Rep. Germany		     	    jb at cs.bonn.edu
			  	http://www-dbv.informatik.uni-bonn.de
---------------------------------------------------------------------