"Daniel L. Silver": PhD thesis available: Selective Transfer of Neural Network Task Knowledge

[Daniel L. Silver]

Dear Connectionists,  

This is to announce the availability of my PhD thesis for download.

Title: Selective Transfer of Neural Network Task Knowledge

Postscript URL:  http://dragon.acadiau.ca/~dsilver/DLS_THESIS.PS
Zip of PS   URL: http://dragon.acadiau.ca/~dsilver/DLS_THESIS.zip
(323 pages) 

Best regards,
    Danny Silver
==============

Keywords:  

task knowledge transfer, artificial neural networks, sequential 
learning, inductive bias, task relatedness, knowledge based 
inductive learning, learning to learn, knowledge consolidation

Abstract:

Within the context of artificial neural networks 
(ANN), we explore the question: How can a 
learning system retain and use previously 
learned knowledge to facilitate future learning?
The research objectives are to develop a 
theoretical model and test a prototype system 
which sequentially retains ANN task knowledge 
and selectively uses that knowledge to bias the 
learning of a new task in an efficient and 
effective manner.
           
A theory of {\em selective functional transfer} 
is presented that requires a learning algorithm 
that employs a {\em measure of task relatedness}.
$\eta$MTL is introduced as a knowledge based 
inductive learning method that learns one or 
more secondary tasks within a back-propagation 
ANN as a source of inductive bias for a primary 
task. $\eta$MTL employs a separate learning 
rate, $\eta_k$, for each secondary task output 
$k$. $\eta_k$ varies as a function of a measure 
of relatedness, $R_k$, between the $k^{th}$ 
secondary task and the primary task of interest.
Three categories of {\em a priori} measures of 
relatedness are developed for controlling 
inductive bias.

The {\em task rehearsal method} (TRM) is 
introduced to address the issue of sequential 
retention and generation of learned task 
knowledge.  The representations of successfully 
learned tasks are stored within a {\em domain 
knowledge} repository.  {\em Virtual training 
examples} generated from domain knowledge are 
rehearsed as secondary tasks in parallel
with each new task using either standard 
multiple task learning (MTL) or $\eta$MTL.

TRM using $\eta$MTL is tested as a method of 
selective knowledge transfer and sequential 
learning on two synthetic domains and one 
medical diagnostic domain. Experiments show that 
the TRM  provides an excellent method of 
retaining and generating accurate functional 
task knowledge. Hypotheses generated are 
compared statistically to single task learning 
and MTL hypotheses. We conclude that selective 
knowledge transfer with $\eta$MTL develops more 
effective hypotheses but not necessarily with 
greater efficiency. The {\em a priori} measures 
of relatedness demonstrate significant value on 
certain domains of tasks but have difficulty 
scaling to large numbers of tasks.  Several 
issues identified during the research indicate 
the importance of consolidating a 
representational form of domain knowledge.



======================================================
Daniel L. Silver        Danny.Silver at AcadiaU.ca
Assistant Professor
Intelligent Information Technology Research Centre 
Jodery School of Computer Science,  Office 315
Acadia University  ph:(902)585-1105 fax:(902)585-1067
Wolfville, NS  B0P 1X0