outlier, robust statistics

[Yong Liu]

Plutowski wrote (Wed, 9 Feb 94)

   >It also points out an appealling definition  of "outlier",
   >My interpretation of this is the following:
   >When the noise variance on the target can depends upon the input 
   >(in statistical jargon, referred to as "heteroscedasticity of
   >the conditional variance of Y_i given X_i")
   >there is the possibility that a plot of the conditional 
   >target variance over the input space could display
   >discontinuous jumps, corresponding to where it is more likely
   >to encounter targets that are much more "noisy" - as compared
   >to targets for neighboring inputs.   Is this accurate?

Yes. It is the heuristics behind modelling the error as a mixture of
normal distributions in (Liu 94). In simple words, the statistical
formulation  regards the error for each data points as from a normal 
distribution with  different variances, and regard the variances as
missing observations.  By using a prior on the variance and EM
algorithm, one can estimate the variance. It turns out during the
estimation, the EM algorithm looks for the data points that have
larger variances and down-weights those data points. 

This way of modelling is in agreement with Dr. Sejnowski's view

  >One man's outlier is another man's data point.  Another
  >way to handle outliers is not to remove them but to model them
  >explicitly. ...


Plutowski also wrote (Wed, 9 Feb 94)

   >I look forward to reading (Liu 94).  Can you (or anyone else)
   >point me to other references utilizing a similar definition
   >of "outlier?"  (IMHO) "outlier" is quite a value-laden term
   >that I tend to avoid since I feel it has multiple and
   >often ambiguous interpretations/definitions.  

Box and Tiao (1968) hold similar views.
Outlier are generated from  a distribution
that is a perturbation to the underlying distribution, for example,
a small amount of noise with ever changing distribution in the
background. Huber's (1981) book is referred as a excellent reference.
Anyway, no matter what outlier is, what one really want is to use a
model/method that is not sensitive to them and predict the relevant
information.

References

Box, G.E.P. and Tiao, G.C.(1968) A Bayesian approach to some outlier
problem. Biometrika, 55, 119-129

Huber (1981) Robust Statistics. John Wiley & Sons, Inc..

BTW. I will be a Phd only three month later. 
-------
Yong Liu
Box 1843
Department of Physics
Institute for Brain and Neural Systems
Brown University
Providence, RI 02912