From schneide at cs.cmu.edu  Wed Jun  1 17:08:44 2016
From: schneide at cs.cmu.edu (Jeff Schneider)
Date: Wed, 1 Jun 2016 17:08:44 -0400
Subject: Auton Lab Thesis Oral - Xuezhi Wang - June 6, 2016
In-Reply-To: <12281c19-5ae7-4df6-0e2c-5d02f1caa380@cs.cmu.edu>
References: <12281c19-5ae7-4df6-0e2c-5d02f1caa380@cs.cmu.edu>
Message-ID: <574F4EDC.3080903@cs.cmu.edu>

Hi guys,

Come and see the next Auton Lab PhD defense on Monday!

Jeff.



-------- Forwarded Message --------
Subject: Thesis Oral - Xuezhi Wang - June 6, 2016
Date: Thu, 26 May 2016 10:13:13 -0400
From: Deborah Cavlovich <deb at cs.cmu.edu>
To: cs-friends at cs.cmu.edu, cs-students at cs.cmu.edu, cs-visitors at cs.cmu.edu, 
cs-advisors-ext at cs.cmu.edu, Catherine Copetas <copetas+ at cs.cmu.edu>

CALENDAR OF EVENTS

Xuezhi Wang

June 6, 2016

12:00 PM, GHC 6501

Thesis Oral

Title:  Active Transfer Learning

Abstract:

Transfer learning algorithms are used when one has sufficient training
data for one supervised learning task (the source task) but only very
limited training data for a second task (the target task) that is
similar but not identical to the first. These algorithms use varying
assumptions about the similarity between the tasks to carry information
from the source to the target task. Common assumptions are that only
certain specific marginal or conditional distributions have changed
while all else remains the same. Moreover, not much work on transfer
learning has considered the case when a few labels in the test domain
are available. Alternatively, if one has only the target task, but also
has the ability to choose a limited amount of additional training data
to collect, then active learning algorithms are used to make choices
which will most improve performance on the target task. These algorithms
may be combined into active transfer learning, but previous efforts have
had to apply the two methods in sequence or use restrictive transfer
assumptions.

This thesis focuses on active transfer learning under the model shift
assumption. We start by proposing two transfer learning algorithms that
allow changes in all marginal and conditional distributions but assume
the changes are smooth in order to achieve transfer between the tasks.
We then propose an active learning algorithm that yields a combined
active transfer learning algorithm. We show that the generalization
error bound of the proposed algorithms is $O(\frac{1}{\lambda_*
\sqrt{n_l}})$ with respect to the labeled target sample size $n_l$,
modified by the smoothness of the change ($\lambda_*$) across domains.
Furthermore, we consider a general case where both the support and the
model change across domains.

On the other hand, multi-task learning attempts to simultaneously
leverage data from multiple domains in order to estimate related
functions on each domain. Similar to transfer learning, multi-task
problems are also solved by imposing some kind of ``smooth" relationship
among/between tasks. We study how different smoothness assumptions on
task relations affect the upper bounds of algorithms proposed for these
problems under different settings. Finally, we propose methods for
transfer learning on distributions.

We demonstrate our methods on both synthetic examples and real-world
applications, including yield estimation on the grape image dataset,
predicting air-quality from Weibo posts for cities, predicting the
probability of success for robot-climbing-obstacle tasks, and
grade-prediction for schools. On all these datasets, our proposed
methods lead to significantly improved prediction performance.

Thesis Committee:
Jeff Schneider, Chair
Christos Faloutsos
Geoff Gordon
Jerry Zhu, University of Wisconsin-Madison




From schneide at cs.cmu.edu  Mon Jun  6 11:40:21 2016
From: schneide at cs.cmu.edu (Jeff Schneider)
Date: Mon, 6 Jun 2016 11:40:21 -0400
Subject: Fwd: Auton Lab Thesis Oral - Xuezhi Wang - noon, June 6, 2016
In-Reply-To: <574F4EDC.3080903@cs.cmu.edu>
References: <574F4EDC.3080903@cs.cmu.edu>
Message-ID: <57559965.2000004@cs.cmu.edu>

Reminder, Xuezhi's defense begins in about 20 min.

Jeff.



-------- Forwarded Message --------
Subject: Auton Lab Thesis Oral - Xuezhi Wang - June 6, 2016
Date: Wed, 1 Jun 2016 17:08:44 -0400
From: Jeff Schneider <schneide at cs.cmu.edu>
To: research at autonlab.org

Hi guys,

Come and see the next Auton Lab PhD defense on Monday!

Jeff.



-------- Forwarded Message --------
Subject: Thesis Oral - Xuezhi Wang - June 6, 2016
Date: Thu, 26 May 2016 10:13:13 -0400
From: Deborah Cavlovich <deb at cs.cmu.edu>
To: cs-friends at cs.cmu.edu, cs-students at cs.cmu.edu, cs-visitors at cs.cmu.edu,
cs-advisors-ext at cs.cmu.edu, Catherine Copetas <copetas+ at cs.cmu.edu>

CALENDAR OF EVENTS

Xuezhi Wang

June 6, 2016

12:00 PM, GHC 6501

Thesis Oral

Title:  Active Transfer Learning

Abstract:

Transfer learning algorithms are used when one has sufficient training
data for one supervised learning task (the source task) but only very
limited training data for a second task (the target task) that is
similar but not identical to the first. These algorithms use varying
assumptions about the similarity between the tasks to carry information
from the source to the target task. Common assumptions are that only
certain specific marginal or conditional distributions have changed
while all else remains the same. Moreover, not much work on transfer
learning has considered the case when a few labels in the test domain
are available. Alternatively, if one has only the target task, but also
has the ability to choose a limited amount of additional training data
to collect, then active learning algorithms are used to make choices
which will most improve performance on the target task. These algorithms
may be combined into active transfer learning, but previous efforts have
had to apply the two methods in sequence or use restrictive transfer
assumptions.

This thesis focuses on active transfer learning under the model shift
assumption. We start by proposing two transfer learning algorithms that
allow changes in all marginal and conditional distributions but assume
the changes are smooth in order to achieve transfer between the tasks.
We then propose an active learning algorithm that yields a combined
active transfer learning algorithm. We show that the generalization
error bound of the proposed algorithms is $O(\frac{1}{\lambda_*
\sqrt{n_l}})$ with respect to the labeled target sample size $n_l$,
modified by the smoothness of the change ($\lambda_*$) across domains.
Furthermore, we consider a general case where both the support and the
model change across domains.

On the other hand, multi-task learning attempts to simultaneously
leverage data from multiple domains in order to estimate related
functions on each domain. Similar to transfer learning, multi-task
problems are also solved by imposing some kind of ``smooth" relationship
among/between tasks. We study how different smoothness assumptions on
task relations affect the upper bounds of algorithms proposed for these
problems under different settings. Finally, we propose methods for
transfer learning on distributions.

We demonstrate our methods on both synthetic examples and real-world
applications, including yield estimation on the grape image dataset,
predicting air-quality from Weibo posts for cities, predicting the
probability of success for robot-climbing-obstacle tasks, and
grade-prediction for schools. On all these datasets, our proposed
methods lead to significantly improved prediction performance.

Thesis Committee:
Jeff Schneider, Chair
Christos Faloutsos
Geoff Gordon
Jerry Zhu, University of Wisconsin-Madison