PhD thesis available

[Tirthankar Raychaudhuri]

The following PhD thesis is now electronically available

ftp://ftp.mpce.mq.edu.au/pub/comp/papers/raychaudhuri.phd_thesis.97.ps (2581 k)
ftp://ftp.mpce.mq.edu.au/pub/comp/papers/raychaudhuri.phd_thesis.97.ps.Z (848 k)

SEEKING THE VALUABLE DOMAIN - QUERY LEARNING IN A COST-OPTIMAL PERSPECTIVE

by Tirthankar RayChaudhuri
   School of Mathematics, Physics, Computing
   and Electronics, Macquarie University, Sydney, NSW 2109
   AUSTRALIA

tirthank at mpce.mq.edu.au
http://www.comp.mq.edu.au/~tirthank

Abstract
--------

Purpose

This thesis is intended as a contribution to the 
theories and principles of active learning and dual control.
It is an account of an investigation which essentially 
examines the problem of how a machine learning system 
can be designed so as to acquire useful data actively,  
i.e., with environment interaction and simultaneously perform cost-effectively.
The primary aim of the research has been to
evolve the general basis for developing an engineering solution for
a self-designing system of the future. In other words,
while developed theory and experimental investigations to 
substantiate the theory are presented in this dissertation, 
the learning system used herein
is based on certain specific assumptions with a view to achieving 
a balance between extremely general theoretical abstraction 
and a practical method that addresses clearly-defined problem parameters. 
The outcome is a strategy of `learning while performing' - consisting
of a set of algorithms directed to addressing a real world scenario 
such as an industrial manufacturing plant from which continuous data 
is available.

Content

The basic learning model used is the neural network function approximator 
trained by a supervised method.
The overall theoretical framework is a system that is endowed with the 
capability of asking questions (querying) in order to obtain the most 
useful information from an environment such as a producing plant.
Instead of modeling system behaviour with a state-space description,
the emphasis is on significant input-output data gathered from a 
system with unknown internal behaviour (black-box).
A locally-optimal or `greedy' strategy 
is used. Although a general theory for multidimensional systems 
has been developed the experimental studies are concentrated within a finite
boundary, two-dimensional, continuous data domain.
This is primarily for the purpose of easy visualisation of results
and computational efficiency. 
The basic principle propounded is that the notion of dollar value 
of a particular output ought to be incorporated within the querying criterion.
In the process of developing such a querying criterion a novel method 
of data subsampling/statistical jack-knifing is applied to Seung and Freund's 
query-by-committee query-by-committee philosophy.
This technique is demonstrated to be highly effective in gathering 
both significant and adequate data samples for accurately modeling
functions with sharp transition points and nonlinear functions.
An extension of the strategy for addressing noisy functions leads to 
derivation of an estimate of the distribution of the true label (output) for
a particular input of the black-box. While such a distribution is 
closely related to the developed querying criterion for information gain, 
it is taken further ahead in the investigation and used in
defining the expected value of a label as an exploitation objective - in a 
cost-benefit analysis perspective. A confidence interval
on this expected value is then analytically derived.
The size of the confidence interval determines the amount of 
exploration by the learner. Experimental studies with 
unimodal output value characteristics reveal differences 
in the performance of the proposed method upon monotonic and 
non-monotonic environments. 
Such performance is mostly based upon a long-term measure of 
yield experimentally computed within a particular finite 
querying boundary.
Two overall approaches to examining the problem are studied - one uses 
the estimated distribution of a label
to compute its expected value (called IMDV: indirectly modeled dollar value), 
the other estimates expected values by applying the
jack-knifed committee method directly to dollar value labels and is 
called a DMDV (directly modeled dollar value) algorithm.
The inherent weakness of the second approach, despite
its obvious computational advantage over the other, is pointed out.
It is then demonstrated that a theoretically more sound but very 
computationally expensive strategy of combining
the strengths of the IMDV and DMDV approaches can result in a technique 
known as the combined modeling of dollar values (CMDV).
Finally it is discussed how the theoretical premise of 
a dual controller is inherent in the querying philosophy proposed. 
This kind of dual control is observer-free and is 
contrasted with more conventional control methods. 
--------------------------------------------------------------------------
Dr Tirthankar RayChaudhuri  	Phone  61 -2 -850-9543 
Department of Computing                               
School of MPCE           	Fax    61 -2 -850-9551
Macquarie University     	E-mail tirthank at mpce.mq.edu.au 
Sydney,NSW 2109          	WWW    http://www-comp.mpce.mq.edu.au/~tirthank