Paper available by ftp

[bishopc]

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/bishop.mixture*.ps.Z 

The following technical report is available by anonymous ftp.

------------------------------------------------------------------------

                        MIXTURE DENSITY NETWORKS

                             Chris M Bishop


             Neural Computing Research Group Report: NCRG/4288

                      Neural Computing Research Group
                            Aston University
                         Birmingham, B4 7ET, U.K.

                      email: c.m.bishop at aston.ac.uk


                               Abstract

In this paper we introduce a general technique for modelling conditional
probability density functions, by combining a mixture distribution model 
with a standard feedforward network. The conventional technique of
minimizing a sum-of-squares or cross-entropy error function leads to 
network outputs which approximate the conditional averages of the target
data, conditioned on the input vector. For classifications problems, with 
a suitably chosen target coding scheme, these averages represent the 
posterior probabilities of class membership, and so can be regared as 
optimal. For problems involving the prediction of continuous variables, 
however, the conditional averages provide only a very limited description 
of the properties of the target variables. This is particularly true for 
problems in which the mapping to be learned is multi-valued, as often 
arises in the solution of inverse problems, since the average of several 
correct target values is not necessarily itself a correct value. In order 
to obtain a complete description of the data, for the purposes of 
predicting the outputs corresponding to new input vectors, we must model 
the conditional probability distribution of the target data, again 
conditioned on the input vector. In this paper we introduce a new class 
of network models obtained by combining a conventional neural network with 
a mixture density model. The complete system is called a Mixture Density 
Network, and can in principle represent arbitrary conditional probability 
distributions in the same way that a conventional neural network can 
represent arbitrary non-linear functions. We demonstrate the effectiveness 
of Mixture Density Networks using both a simple 1-input 1-output mapping, 
and a problem involving robot inverse kinematics.

-------------------------------------------------------------------- 
ftp instructions: 

This paper is split into two files to keep the uncompressed 
postscript files below 2Mb.

  bishop.mixture1.ps.Z   (size 445839)  pages 1 -- 16
  bishop.mixture2.ps.Z   (size 364598)  pages 17 -- 25

% ftp archive.cis.ohio-state.edu
Name: anonymous
password: your full email address
ftp> cd pub/neuroprose
ftp> binary
ftp> get bishop.mixture1.ps.Z
ftp> get bishop.mixture2.ps.Z
ftp> bye
% uncompress bishop*
% lpr bishop*

-------------------------------------------------------------------- 

  Professor Chris M Bishop           Tel. +44 (0)21 359 3611 x4270
  Neural Computing Research Group    Fax. +44 (0)21 333 6215
  Dept. of Computer Science          c.m.bishop at aston.ac.uk
  Aston University               
  Birmingham B4 7ET, UK

--------------------------------------------------------------------