TR available: Prior Information and Generalized Questions

[Joerg_Lemm]

Dear colleagues,

the following TR is now available:



        "Prior Information and Generalized Questions"
         MIT AI Memo No. 1598 (C.B.C.L paper No. 141)


                       Joerg C. Lemm


                         Abstract

In learning problems available information is usually divided 
into two categories:
examples of function values (or training data) and prior information
(e.g.\ a smoothness constraint).
This paper
1.) studies aspects on which these two categories usually differ,
    like their relevance for generalization and
    their role in the loss function,
2.) presents a unifying formalism, where both types of information are 
    identified with answers to generalized questions,
3.) shows what kind of generalized information
    is necessary to enable learning,
4.) aims to put usual training data and prior information 
    on a more equal footing by discussing possibilities and variants of
    measurement and control for generalized questions,
    including the examples of smoothness and symmetries,
5.) reviews shortly the measurement of linguistic concepts
    based on fuzzy  priors, and principles to combine preprocessors,
6.) uses a Bayesian decision theoretic framework, contrasting
    parallel and inverse decision problems,
7.) proposes, for problems with non--approximation aspects,
    a Bayesian two step approximation consisting
    of posterior maximization and a subsequent risk minimization,
8.) analyses empirical risk minimization
    under the aspect of nonlocal information
9.) compares the Bayesian two step approximation with
    empirical risk minimization,
    including their interpretations of Occam's razor,
10.) formulates examples of stationarity conditions for
    the maximum posterior approximation with nonlocal and nonconvex priors,
    leading to inhomogeneous nonlinear equations,
    similar for example to equations in scattering theory in physics.

In summary, the paper emphasizes the need of empirical measurement and 
control of prior information and of their explicit treatment in theory.


----------------------------------------------------------------------

Comments are welcome!

Download sites:

ftp://publications.ai.mit.edu/ai-publications/1500-1999/AIM-1598.ps
http://planck.uni-muenster.de:8080/~lemm/prior.ps.Z 


=======================================================================
Joerg Lemm
Institute for Theoretical Physics I
Wilhelm-Klemm-Str. 9
D- 48149 Muenster, Germany
Email:     lemm at uni-muenster.de
Home page: http://planck.uni-muenster.de:8080/~lemm/
=======================================================================