NetTools - a package of tools for NN analysis

[stevep@cs.uq.oz.au]

NetTools is a package of analysis tools and a tech. report 
demonstrating two of these techniques.


		Analysis Tools for Neural Networks.

			by Simon Dennis and Steven Phillips

Abstract - A large volume of neural net research in the 1980's
involved applying backpropagation to difficult and generally
poorly understood tasks. Success was sometimes measured on the
ability of the network to replicate the required mapping. The
difficulty with this approach, which is essentially a black
box analysis, is that we are left with little additional 
understanding of the problem or the way in which the neural net
has solved it. Techniques which can look inside the black box
are required. This report focuses on two statistical analysis
techniques (Principal Components Analysis and Canonical 
Discriminant Analysis) as tools for analysing and interpreting
network behaviour in the hidden unit layers.




			Net Tools

The following package contains three tools for network analysis:

	gea - Group Error Analysis
	pca - Principal Components Analysis
	cda - Canonical Discriminants Analysis

TOOL DESCRIPTIONS

Group Error Analysis (gea)

Gea counts errors. It takes an output file and a target file and
optionally a groups file. Each line in the output file is an output
vector and the lines in the targets file are the corresponding correct
values. If all values in the output file are within criterion of the
those in the target file then the pattern is considered correct. Note
that this is a more stringent measure of correctness than the total
sum of squares. In particular it requires the outputs to be either high
or low rather than taking some average intermediate value. If a groups 
file is provided then gea will separate the error count into the groups
provided. 

Principal Components Analysis (pca)

Principle components analysis takes a set of points in a high dimensional space 
and determines the major components of variation. The principal components are
labeled 0-(n-1) where n is the dimensionality of the space (i.e. the
number of hidden units). The original points can be projected onto these
vectors. The result is a low dimensional plot which has hopefully extracted
the important information from the high dimensional space. 

Canonical Discriminants Analysis (cda)

Canonical discriminant analysis takes a set of grouped points in a high
dimensional space and determines the components such that points within
a group form tight clusters. These points are called the canonical
variates and are labeled 0-(n-1) where n is the dimensionality of the
space (i.e. the number of hidden units). The original points can be
projected on to these vectors. The result is a low dimensional plot
which has clustered the points belonging to each group.

TECHNICAL REPORT

Reference: Simon Dennis and Steven Phillips.
	   Analysis Tools for Neural Networks.
	   Technical Report 207,
	   Department of Computer,
	   University of Queensland,
	   Queensland, 4072
	   Australia
	   May, 1991

NetTools.ps is a technical report which demonstrates the results which
can be obtained from pca and cda. It outlines the advantages of each 
and points out some interpretive pitfalls which should be avoided.

TUTORIAL

The directory tute contains a tutorial designed at the University of 
Queensland by Janet Wiles and Simon Dennis to introduce students
to network analysis. It uses the iris data first published
by Fisher in 1936. The backpropagation simulator is tlearn developed at
UCSD by Jeffery Elman and colleagues. In addition the tutorial uses the
hierarchical clustering program, cluster, which was written by Yoshiro Miyata
and modified by Andreas Stolcke.

These tools can be obtained as follows


$ ftp crl.ucsd.edu
Connected to crl.ucsd.edu.
220 crl FTP server (SunOS 4.1) ready.
Name (crl.ucsd.edu:mav): anonymous
331 Guest login ok, send ident as password.
Password:
230 Guest login ok, access restrictions apply.
ftp> cd pub/neuralnets
250 CWD command successful.
ftp> bin
200 Type set to I.
ftp> get NetTools.tar.Z
200 PORT command successful.
150 Binary data connection for NetTools.tar.Z (130.102.64.15,1240) (185900 bytes).
226 Binary Transfer complete.
local: NetTools.tar.Z remote: NetTools.tar.Z
185900 bytes received in 1.9e+02 seconds (0.97 Kbytes/s)
ftp> quit
221 Goodbye.
$ zcat NetTools.tar.Z | tar -xf -

Shalom
       Simon and Steven

-------------------------------------------------------------------------------
Simon Dennis                  Address: Department of Computer Science
Email: mav at cs.uq.oz.au                 University of Queensland
                                       QLD 4072
                                       Australia
-------------------------------------------------------------------------------