Query: Infinite priors

[Stephen M. Omohundro]

> Date: Thu, 4 Jan 1996 09:57:57 +0900
> From: hicks at cs.titech.ac.jp
> 
> 
> I have the following question concerning the existence of certain priors.
> 
> Can we say that a uniform prior over an infinite domain exists?  For example,
> the uniform prior over all natural numbers.  I wonder since it cannot be
> expressed in the form p(n) = (f(n)/\sum_n f(n)), where f(n) is a well defined
> function over the natural numbers.  In general, if f(n) is not a summable
> series, then can the probability function p(n), whose elements have the ratios
> of the elements f(n), i.e., p(n)/p(m) = f(n)/f(m), be said to exist?
> 
> I ask because a true Bayesian approach to some problems may require the prior
> to be defined.  If there is no prior, then we can't say we are taking a
> Bayesian approach.  If an infinite uniform prior does not exist, then we
> cannot take the approach that "no prior knowledge" = "infinite uniform prior".
> I.e., it would imply that any Bayesian approach involving the prior MUST begin
> with some assumptions about the prior (i.e., it must be formed from a
> summable/integrable function).
> 
> References or opinions would be welcome.  Craig Hicks. hicks at cs.titech.ac.jp
> 

These are generally called "improper priors". You can still do much of
the Bayesian paradigm using them because in many situations the
likelihood function is such that the posterior (which is proportional
to the prior times the likelihood) is normalizable even if the prior
isn't. Formally you can treat them using a limiting sequence of proper
priors. Most books on Bayesian analysis have some discussion of this
topic. 

--Steve

-- 
Stephen M. Omohundro               http://www.neci.nj.nec.com/homepages/om
NEC Research Institute, Inc.                        om at research.nj.nec.com
4 Independence Way                                     Phone: 609-951-2719
Princeton, New Jersey 08540                              Fax: 609-951-2488