simulator announcement
Russell Leighton
russ at dash.mitre.org
Wed Mar 6 08:20:35 EST 1991
The following describes a neural network simulation environment
made available free from the MITRE Corporation. The software
contains a neural network simulation code generator which generates
high performance C code implementations for backpropagation networks.
Also included is a graphical interface for visualization.
PUBLIC DOMAIN
NEURAL NETWORK SIMULATOR
AND GRAPHICS ENVIRONMENT
AVAILABLE
Aspirin for MIGRAINES
Version 4.0
The Mitre Corporation is making available free to the public a
neural network simulation environment called Aspirin for MIGRAINES.
The software consists of a code generator that builds neural network
simulations by reading a network description (written in a language
called "Aspirin") and generates a C simulation. A graphical interface
(called "MIGRAINES") is provided for platforms that support the Sun
window system NeWS1.1. For platforms that do not support NeWS1.1
no graphics are currently available.
The system has been ported to a number of platforms:
Sun3/4
Silicon Graphics Iris
IBM RS/6000
DecStation
Cray YMP
Included with the software are "config" files for these platforms.
Porting to other platforms may be done by choosing the "closest"
platform currently supported and adapting the config files.
Aspirin 4.15
------------
The software that we are releasing now is principally for creating,
and evaluating, feed-forward networks such as those used with the
backpropagation learning algorithm. The software is aimed both at
the expert programmer/neural network researcher who may wish to tailor
significant portions of the system to his/her precise needs, as well
as at casual users who will wish to use the system with an absolute
minimum of effort.
Aspirin was originally conceived as ``a way of dealing with MIGRAINES.''
Our goal was to create an underlying system that would exist behind
the graphics and provide the network modeling facilities.
The system had to be flexible enough to allow research, that is,
make it easy for a user to make frequent, possibly substantial, changes
to network designs and learning algorithms. At the same time it had to
be efficient enough to allow large ``real-world'' neural network systems
to be developed.
Aspirin uses a front-end parser and code generators to realize this goal.
A high level declarative language has been developed to describe a network.
This language was designed to make commonly used network constructs simple
to describe, but to allow any network to be described. The Aspirin file
defines the type of network, the size and topology of the network, and
descriptions of the network's input and output. This file may also include
information such as initial values of weights, names of user defined
functions, and hints for the MIGRAINES graphics system.
The Aspirin language is based around the concept of a "black box".
A black box is a module that (optionally) receives input and
(necessarily) produces output. Black boxes are autonomous units
that are used to construct neural network systems. Black boxes
may be connected arbitrarily to create large possibly heterogeneous
network systems. As a simple example, pre or post-processing stages
of a neural network can be considered black boxes that do not learn.
The output of the Aspirin parser is sent to the appropriate code
generator that implements the desired neural network paradigm.
The goal of Aspirin is to provide a common extendible front-end language
and parser for different network paradigms. The publicly available software
will include a backpropagation code generator that supports several
variations of the backpropagation learning algorithm. For backpropagation
networks and their variations, Aspirin supports a wide variety of
capabilities:
1. feed-forward layered networks with arbitrary connections
2. ``skip level'' connections
3. one and two-dimensional tessellations
4. a few node transfer functions (as well as user defined)
5. connections to layers/inputs at arbitrary delays,
also "Waibel style" time-delay neural networks
The file describing a network is processed by the Aspirin parser and
files containing C functions to implement that network are generated.
This code can then be linked with an application which uses these
routines to control the network. Optionally, a complete simulation
may be automatically generated which is integrated with the graphics
and can read data in a variety of file formats. Currently supported
file formats are:
Ascii
Type1, Type2, Type3 (simple floating point file formats)
ProMatlab
Examples
--------
A set of examples comes with the distribution:
xor: from RumelHart and McClelland, et al,
"Parallel Distributed Processing, Vol 1: Foundations",
MIT Press, 1986, pp. 330-334.
encode: from RumelHart and McClelland, et al,
"Parallel Distributed Processing, Vol 1: Foundations",
MIT Press, 1986, pp. 335-339.
detect: Detecting a sine wave in noise.
characters: Learing to recognize 4 characters independent
of rotation.
sonar: from Gorman, R. P., and Sejnowski, T. J. (1988).
"Analysis of Hidden Units in a Layered Network Trained to
Classify Sonar Targets" in Neural Networks, Vol. 1, pp. 75-89.
spiral: from Kevin J. Lang and Michael J, Witbrock, "Learning
to Tell Two Spirals Apart", in Proceedings of the 1988 Connectionist
Models Summer School, Morgan Kaufmann, 1988.
ntalk: from Sejnowski, T.J., and Rosenberg, C.R. (1987).
"Parallel networks that learn to pronounce English text" in
Complex Systems, 1, 145-168.
perf: A large network used only for performance testing.
Performance of Aspirin simulations
----------------------------------
The backpropagation code generator produces simulations
that run very efficiently. Aspirin simulations do
best on vector machines when the networks are large,
as exemplified by the Cray's performance. All simulations
were done using the Unix "time" function and include all
simulation overhead. The connections per second rating was
calculated by multiplying the number of iterations by the
total number of connections in the network and dividing by the
"user" time provided by the Unix time function. Two tests were
performed. In the first, the network was simply run "forward"
100,000 times and timed. In the second, the network was timed
in learning mode and run until convergence. Under both tests
the "user" time included the time to read in the data and initialize
the network.
Sonar:
This network is a two layer fully connected network
with 60 inputs: 2-34-60.
Millions of Connections per Second
Forward:
SparcStation1: 1
IBM RS/6000 320: 2.8
Cray YMP: 15.7
Backward:
SparcStation1: 0.3
IBM RS/6000 320: 0.8
Cray YMP: 7
Gorman, R. P., and Sejnowski, T. J. (1988). "Analysis of Hidden Units
in a Layered Network Trained to Classify Sonar Targets" in Neural Networks,
Vol. 1, pp. 75-89.
Nettalk:
This network is a two layer fully connected network
with [29 x 7] inputs: 26-[15 x 8]-[29 x 7]
Millions of Connections per Second
Forward:
SparcStation1: 1
IBM RS/6000 320: 3.5
Cray YMP: 64
Backward:
SparcStation1: 0.4
IBM RS/6000 320: 1.3
Cray YMP: 24.8
Sejnowski, T.J., and Rosenberg, C.R. (1987). "Parallel networks that
learn to pronounce English text" in Complex Systems, 1, 145-168.
Perf:
This network was only run on the Cray. It is very large
with very long vectors. The performance on this network
is in some sense a peak performance for a machine.
This network is a two layer fully connected network
with 2048 inputs: 128-512-2048
Millions of Connections per Second
Forward:
Cray YMP: 96.3
Backward:
Cray YMP: 18.9
Note: The cray benchmarks are courtesy of the Center for
High Performance Computing at the University of Texas.
Aspirin 5.0
-----------
The next release of the software *may* include:
1. 2nd order (quadratic) connections
2. Auto-regressive nodes (this a form
of limited recurrence)
3. Code generators for other (not backprop) neural network
learning algorithms.
4. More supported file formats
5. More config files for other platforms.
MIGRAINES 4.0
-------------
MIGRAINES is a graphics system for visualizing
neural network systems. The graphics that are currently being
released are exclusively for feed-forward networks. They provide
the ability to display networks, arc weights, node values, network
inputs, network outputs, and target outputs in a wide variety of formats.
There are many different representations that may be used to
display arcs weights and node values, including
pseudo-color (or grayscale) arrays (with user modifiable colors
and value-to-color mappings), various plots, bar charts and
other pictorial representations.
MIGRAINES is not necessary for the execution of the Aspirin system.
Networks may be designed, executed, tested, and saved entirely apart from
any graphic interface. The more exotic the network being
considered, the smaller the amount of graphics that will be useful.
However, the graphics offer such a degree of creative and
analytic power for neural network research that even the most
jaded researcher will find them useful. Although the graphics
were developed for the NeWS1.1 window system, it can be run under
Sun's OpenWindows which supports NeWS applications. Note:
OpenWindows is not 100% backward compatible with NeWS1.1 so
some features of the graphics may not work well.
MIGRAINES 5.0
-------------
The next release will replace the NeWS1.1 graphics with
an X based system as well extending the graphical capabilities.
An interface to the scientific visualization system apE2.0
*may* be available.
How to get the software
-----------------------
The software is available from two FTP sites, CMU's simulator
collection and UCLA's cognitive science machines. The compressed tar
file is a little more than 2 megabytes. The software is currently
only available via anonymous FTP.
> To get the software from CMU's simulator collection:
1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu"
(128.2.254.155).
2. Log in as user "anonymous" with password your username.
3. Change remote directory to "/afs/cs/project/connect/code". Any
subdirectories of this one should also be accessible. Parent directories
should not be.
4. At this point FTP should be able to get a listing of files in this
directory and fetch the ones you want.
Problems? - contact us at "connectionists-request at cs.cmu.edu".
5. Set binary mode by typing the command "binary" ** THIS IS IMPORTANT **
6. Get the file "am4.tar.Z"
> To get the software from UCLA's cognitive science machines:
1. Create an FTP connection to "polaris.cognet.ucla.edu" (128.97.50.3)
(typically with the command "ftp 128.97.50.3")
2. Log in as user "anonymous" with password your username.
3. Change remote directory to "alexis", by typing the command "cd alexis"
4. Set binary mode by typing the command "binary" ** THIS IS IMPORTANT **
5. Get the file by typing the command "get am4.tar.Z"
How to unpack the software
--------------------------
After ftp'ing the file make the directory you
wish to install the software. Go to that
directory and type:
zcat am4.tar.Z | tar xvf -
How to print the manual
-----------------------
The user manual is located in ./doc in a
few compressed PostScript files. To print
each file on a PostScript printer type:
zcat <file.ps.Z> | lpr
Thanks
------
Thanks to the folks at CMU and UCLA for the ftp site.
Thanks to the folks at the Center for High Performance
Computing at the University of Texas for the use
of their computers.
Copyright and license agreement
-------------------------------
Since the Aspirin/MIGRAINES system is licensed free of charge,
the MITRE Corporation provides absolutely no warranty. Should
the Aspirin/MIGRAINES system prove defective, you must assume
the cost of all necessary servicing, repair or correction.
In no way will the MITRE Corporation be liable to you for
damages, including any lost profits, lost monies, or other
special, incidental or consequential damages arising out of
the use or in ability to use the Aspirin/MIGRAINES system.
This software is the copyright of The MITRE Corporation.
It may be freely used and modified for research and development
purposes. We require a brief acknowledgement in any research
paper or other publication where this software has made a significant
contribution. If you wish to use it for commercial gain you must contact
The MITRE Corporation for conditions of use. The MITRE Corporation
provides absolutely NO WARRANTY for this software.
February, 1991
Russell Leighton
Alexis Wieland
The MITRE Corporation
7525 Colshire Dr.
McLean, Va. 22102-3481
More information about the Connectionists
mailing list