new paper from JMLR: Task Clustering and Gating for Bayesian Multitask Learning

[David 'Pablo' Cohn]

The Journal of Machine Learning Research (www.jmlr.org) is pleased to
announce publication of the fifth paper in Volume 4:

--------------------------

Task Clustering and Gating for Bayesian Multitask Learning

Bart Bakker and Tom Heskes
JMLR 4(May):83-99, 2003

Abstract

Modeling a collection of similar regression or classification tasks can
be improved by making the tasks 'learn from each other'. In machine
learning, this subject is approached through 'multitask learning', where
parallel tasks are modeled as multiple outputs of the same network. In
multilevel analysis this is generally implemented through the
mixed-effects linear model where a distinction is made between 'fixed
effects', which are the same for all tasks, and 'random effects', which
may vary between tasks. In the present article we will adopt a Bayesian
approach in which some of the model parameters are shared (the same for
all tasks) and others more loosely connected through a joint prior
distribution that can be learned from the data. We seek in this way to
combine the best parts of both the statistical multilevel approach and
the neural network machinery.

The standard assumption expressed in both approaches is that each task
can learn equally well from any other task. In this article we extend
the model by allowing more differentiation in the similarities between
tasks. One such extension is to make the prior mean depend on
higher-level task characteristics. More unsupervised clustering of tasks
is obtained if we go from a single Gaussian prior to a mixture of
Gaussians. This can be further generalized to a mixture of experts
architecture with the gates depending on task characteristics.

All three extensions are demonstrated through application both on an
artificial data set and on two real-world problems, one a school problem
and the other involving single-copy newspaper sales.

----------------------------------------------------------------------------
This paper is available electronically at http://www.jmlr.org in
PostScript and PDF formats. The papers of Volumes 1, 2 and 3 are also
available electronically from the JMLR website, and in hardcopy from the
MIT Press; please see http://mitpress.mit.edu/JMLR for details.

-David Cohn, <david.cohn at acm.org>