[arthur@mail4.ai.univie.ac.at: TR: Limitations of SOM]

[Shimon Edelman]

Seeing that comments are welcome... there seems to be a rather glaring
gap in the references in this TR. Fukunaga proposed a similar
combination of clustering and topology-preservation criteria in 1972,
and there was a recent paper by Webb following up on that work. 

It would have been nice to see Baxter's idea of Canonical Vector
Quantization discussed in this context.

By the way, what is called MDS in this TR is actually not (although it
is related; MDS is the process of placement of points in a metric
space in a maner that preserves their distances - or the ranks thereof
- without the knowledge of the original coordinates of the points).

@Article{KoontzFukunaga72,
  author = 	 "W. L. G. Koontz and K. Fukunaga",
  title = 	 "A nonlinear feature extraction algorithm using
		  distance information",
  journal =	 "IEEE Trans. Comput.",
  year =	 1972,
  volume =	 21,
  pages =	 "56-63",
  annote =	 "combines class separation and distance preservation
		  criteria for dimensionality reduction"
}

@Article{Webb95,
  author = 	 "A. R. Webb",
  title = 	 "Multidimensional-Scaling by Iterative Majorization
		  Using Radial Basis Functions",
  journal =	 "Pattern Recognition",
  year =	 1995,
  volume =	 28,
  pages =	 "753-759",
  annote =	 "MDS, RBFs, nonlinear PCA. This paper considers the
		  use of radial basis functions for modelling the
		  non-linear transformation of a data set obtained by
		  a multidimensional scaling analysis. This approach
		  has two advantages over conventional nonmetric
		  multidimensional scaling. It reduces the number of
		  parameters to estimate and it provides a
		  transformation that may be used on an unseen test
		  set. A scheme based on iterative majorization is
		  proposed for obtaining the parameters of the network."
}

@TechReport{Baxter95b,
  author = 	 "J. Baxter",
  title = 	 "The Canonical Metric for Vector Quantization",
  institution =  "University of London",
  year = 	 1995,
  type =	 "NeuroCOLT",
  number =	 "NC-TR-95-047",
  annote =	 "Abstract. To measure the quality of a set of vector
		  quantization points a means of measuring the
		  distance between two points is required. Common
		  metrics such as the {Hamming} and {Euclidean}
		  metrics, while mathematically simple, are 
		  inappropriate for comparing speech signals or
		  images.  In this paper it is argued that there often
		  exists a natural {environment} of functions to
		  the quantization process (for example, the word
		  classifiers in speech recognition and the character
		  classifiers in character recognition) and that such
		  an enviroment induces a {canonical metric} on
		  the space being quantized.  It is proved that
		  optimizing the {reconstruction error} with
		  respect to the canonical metric gives rise to
		  optimal approximations of the functions in the
		  environment, so that the canonical metric can be
		  viewed as embodying all the essential information
		  relevant to learning the functions in the
		  environment.  Techniques for {learning} the
		  canonical metric are discussed, in particular the
		  relationship between learning the canonical metric
		  and {internal representation learning}"
}


-Shimon

Dr. Shimon Edelman,  Center for Biol & Comp Learning, MIT
(on leave from Weizmann Inst of Science, Rehovot, Israel)
Web home:      http://eris.wisdom.weizmann.ac.il/~edelman
fax: (+1) 617 253-2964  tel: 253-0549  edelman at ai.mit.edu

> From: Arthur Flexer <arthur at mail4.ai.univie.ac.at>
> Subject: TR: Limitations of SOM
> To: connectionists at cs.cmu.edu
> Date: Mon, 9 Sep 1996 19:04:10 +0200 (MET DST)
> 
> Dear colleagues,
> 
> the following report is available via my personal WWW-page:
> 
> 	http://www.ai.univie.ac.at/~arthur/
> as
> 	ftp://ftp.ai.univie.ac.at/papers/oefai-tr-96-23.ps.Z
> 
> Sorry, there are no hardcopies available, comments are welcome!
> 
> 	Sincerely, Arthur Flexer
> 
> -----------------------------------------------------------------------------
> Arthur Flexer					       arthur at ai.univie.ac.at
> Austrian Research Inst. for Artificial Intelligence    +43-1-5336112(Tel)
> Schottengasse 3, A-1010 Vienna, Austria		       +43-1-5320652(Fax) 
> -----------------------------------------------------------------------------
> 
> 
> Flexer A.: Limitations of self-organizing maps for vector quantization and
>    multidimensional scaling, to appear in: Advances in Neural Information
>    Processing Systems 9, edited by M.C. Mozer, M.I. Jordan, and T. Petsche,
>    available in 1997.
> 
> Abstract:
> 
>   The limitations of using self-organizing maps (SOM) for either
>   clustering/vector quantization (VQ) or multidimensional scaling
>   (MDS) are being discussed by reviewing recent empirical findings and
>   the relevant theory. SOM's remaining ability of doing both VQ {\em
>   and} MDS at the same time is challenged by a new combined
>   technique of adaptive {\em K}-means clustering plus Sammon mapping
>   of the cluster centroids. SOM are shown to perform significantly
>   worse in terms of quantization error, in recovering the structure of
>   the clusters and in preserving the topology in a comprehensive
>   empirical study using a series of multivariate normal clustering
>   problems.
> 
>