Tesis + Papers Announcement

[Nathalie Japkowicz]

Dear Connectionists,

I am pleased to announce the availability of my Ph.D. Dissertation
and of a few related papers.

Regards,
Nathalie.

-----------------------------------------------------------------------
Thesis:
-------

Title: 	  "Concept-Learning in the Absence of Counter-Examples:
       	  An Autoassociation-Based Approach to Classification"

Advisors: Stephen Jose Hanson and Casimir A. Kulikowski

URL:	  http://borg.cs.dal.ca/~nat/Research/thesis.ps.gz


Abstract:
--------

The overwhelming majority of research currently pursued within the framework 
of concept-learning concentrates on discrimination-based learning. 
Nevertheless, this emphasis can present a practical problem: there are 
real-world engineering problems for which counter-examples are both scarce 
and difficult to gather. For these problems, recognition-based learning
systems are much more appropriate because they do not use counter-examples 
in the concept-learning phase and thus require fewer counter-examples 
altogether. The purpose of this dissertation is to analyze a promising 
connectionist recognition-based learning system--- autoassociation-based 
classification---and answer the following questions raised by a preliminary
comparison of the autoassociator and its discrimination counterpart, the 
Multi-Layer Perceptron (MLP), on three real-world domains: 

     * What features of the autoassociator make it capable of performing 
       classification in the absence of counter-examples? 
     * What causes the autoassociator to be significantly more efficient 
       than MLP in certain domains? 
     * What domain characteristics cause the autoassociator to be more 
       accurate than MLP and MLP to be more accurate than the autoassociator? 

A study of the two systems in the context of these questions yields the 
conclusions that 1) Autoassociation-based classification is possible in a 
particular class of practical domains called non-linear and multi-modal 
because the autoassociator uses a multi-modal specialization bias to 
compensate for the absence of counter-examples. This bias can be controlled 
by varying the capacity of the autoassociator. 2) The difference in efficiency
between the autoassociator and MLP observed on this class of domains is caused
by the fact that the autoassociator uses a (fast) bottom-up generalization 
strategy whereas MLP has recourse to a (slow) top-down one, despite the fact 
that the two systems are both trained by the backpropagation procedure. 3) The
autoassociator classifies more accurately than MLP domains requiring 
particularly strong specialization biases caused by the counter-conceptual 
class or particularly weak specialization biases caused by the conceptual 
class. However, MLP is more accurate than the autoassociator on domains 
requiring particularly strong specialization biases caused by the conceptual 
class. 

The results of this study thus suggest that recognition-based learning, 
which is often dismissed in favor of discrimination-based ones in the 
context of concept-learning, may present an interesting array of 
classification strengths. 

------------------------------------------------------------------------

Related Papers:
---------------

* "Nonlinear Autoassociation is not Equivalent to PCA" , Japkowicz, N., 
   Hanson S.J., and Gluck, M.A. in Neural Computation (in press).  

   Abstract:
   ---------
   A common misperception within the Neural Network community is that even 
   with nonlinearities in their hidden layer, autoassociators trained with 
   Backpropagation are equivalent to linear methods such as Principal 
   Component Analysis (PCA). The purpose of this paper is to demonstrate that 
   nonlinear autoassociators actually behave differently from linear methods 
   and that they can outperform these methods when used for latent extraction, 
   projection and classification. While linear autoassociators emulate PCA and 
   thus exhibit a flat or unimodal reconstruction error surface, 
   autoassociators with nonlinearities in their hidden layer learn domains by 
   building error reconstruction surfaces that, depending on the task, contain 
   multiple local valleys. This particular interpolation bias allows nonlinear 
   autoassociators to represent appropriate classifications of nonlinear 
   multi-modal domains, in contrast to linear autoassociators which are 
   inappropriate for such tasks. In fact, autoassociators with hidden unit
   nonlinearities can be shown to perform nonlinear classification and 
   nonlinear recognition. 

   URL: http://borg.cs.dal.ca/~nat/Papers/neuralcomp.ps.gz

* "Adaptability of the Backpropagation Procedure" , Japkowicz, N. and 
  Hanson S.J., in the proceedings of the 1999 International Joint 
  Conference in Neural Networks (IJCNN-99) .  

  Abstract:
  ---------
  Possible paradigms for concept learning by feedforward neural networks 
  include discrimination and recognition. An interesting aspect of this 
  dichotomy is that the recognition-based implementation can learn certain 
  domains much more efficiently than the discrimination-based one, despite 
  the close structural relationship between the two systems. The purpose of 
  this paper is to explain this difference in efficiency. We suggest that it 
  is caused by a difference in the generalization strategy adopted by the 
  Backpropagation procedure in both cases: while the autoassociator uses a 
  (fast) bottom-up strategy, MLP has recourse to a (slow) top-down one, 
  despite the fact that the two systems are both optimized by the 
  Backpropagation procedure. This result is important because it sheds some 
  light on the nature of Backpropagation's adaptative capability.  From a 
  practical viewpoint, it suggests a deterministic way to increase the 
  efficiency of Backpropagation-trained feedforward networks.

  URL: http://borg.cs.dal.ca/~nat/Papers/ijcnn-5.ps.gz


* "Are we Better off without Counter Examples" , Japkowicz, N., in the 
  proceedings of the 1999 conference on Advances in Intelligent Data Analysis 
  (AIDA-99).  

  Abstract:
  ---------
  Concept-learning is commonly implemented using discrimination-based 
  techniques which rely on both examples and counter-examples of the concept. 
  Recently, however, a recognition-based approach that learns a concept in 
  the absence of counter-examples was shown to be more accurate than its 
  discrimination counterpart on two real-world domains and as accurate on the 
  third. The purpose of this paper is to find out whether this recognition-
  based approach is generally more accurate than its discrimination 
  counterpart or whether the results it obtained previously are purely 
  coincidental. The analysis conducted in this paper concludes that the 
  results obtained on the real-world domains were not coincidental, and this 
  suggests that recognition-based approaches are promising techniques worth 
  studying in greater depth. 

  URL: http://borg.cs.dal.ca/~nat/Papers/accuracy.ps.gz


* "A Novelty Detection Approach to Classification" , Japkowicz, N., Myers, C. 
  & Gluck, M., in the proceedings of the Fourteenth International Joint 
  Conference on Artificial Intelligence (IJCAI-95). pp. 518-523.  

  Abstract:
  ---------
  Novelty Detection techniques are concept-learning methods that proceed by 
  recognizing positive instances of a concept rather than differentiating 
  between its positive and negative instances. Novelty Detection approaches 
  consequently require very few, if any, negative training instances. This 
  paper presents a particular Novelty Detection approach to classification
  that uses a Redundancy Compression and Non-Redundancy Differentiation 
  technique based on the Gluck & Myers model of the hippocampus, a part of 
  the brain critically involved in learning and memory. In particular, this 
  approach consists of training an autoencoder to reconstruct positive input 
  instances at the output layer and then using this autoencoder to recognize 
  novel instances. Classification is possible, after training, because 
  positive instances are expected to be reconstructed accurately while
  negative instances are not. The purpose of this paper is to compare HIPPO, 
  the system that implements this technique, to C4.5 and feedforward neural 
  network classification on several applications. 

  URL: http://borg.cs.dal.ca/~nat/Papers/ijcai95_final.ps.gz

--
Nathalie Japkowicz, Ph.D.
Assistant Professor
Faculty of Computer Science
DalTech/Dalhousie University
6050 University Avenue
Halifax, Nova Scotia
Canada, B3H 1W5
e-mail: nat at cs.dal.ca
Homepage: http://borg.cs.dal.ca/~nat