ANN fault tol.

[Mike Rudnick]

Below is a synopsis of the references/material I received in response
to an earlier request for pointers to work on the fault tolerance of
artificial neural networks.  Although there has been some work done
relating directly to ANN models, most of the work appears to have been
motivated by VLSI implementation and fault tolerance concerns.

Apparently, and this is speculation on my part, the folklore that
artificial neural networks are fault tolerant derives mostly from the
fact that they resemble biological neural networks, which generally
don't stop working when a few neurons die here and there.

Although it looks like I'm not going to be doing ANN fault tolerance
as my dissertation topic, I can't help but feel this line of research
contains a number of outstanding phd topics.

Mike Rudnick				Computer Science & Eng. Dept.
Domain:	rudnick at cse.ogi.edu		Oregon Graduate Institute (was OGC)
UUCP: {tektronix,verdix}!ogicse!rudnick	19600 N.W. von Neumann Dr.
(503) 690-1121 X7390 (or X7309)		Beaverton, OR. 97006-1999

-----

From: platt at synaptics.com (John Platt)

  Well, one of the original papers about building a neural network in
analog VLSI had a chip where about half of then synapses were broken,
but the chip still worked. Look at

``VLSI Architecutres for Implementation of Neural Networks''
by Massimo A. Sivilotti, Michael Emerling, and Carver A. Mead,
in ``Neural Networks for Computing'', AIP Conference Proceedings 151,
John S. Denker, ed., pp. 408-413

-----

From: Jonathan Mills <rutgers!iuvax.cs.indiana.edu!jwmills>

You might be interested in a paper submitted to the 20th Symposium on
Multiple-Valued Logic titled "Lukasiewicz Logic Arrays", describing work
done by M. G. Beavers, C. A. Daffinger and myself.  These arrays (LLAs
for short) can be used with other circuit components to fabricate neural
nets, expert systems, fuzzy inference engines, sparse distributed memories
and so forth.  They are analog circuits, massively parallel, based on my
work on inference cellular automata, and are inherently fault-tolerant.

In simulations I have conducted, the LLAs produce increasingly noisy
output as individual processors fail, or as groups of processors randomly
experience stuck-at-one and/or stuck-at-zero faults.  While we have
much more work to do, it does appear that with some form of averaging
the output of an LLA can be preserved without noticeable error with up to
one-third of the processors faulty (as long as paths exist from some inputs
to the output).  If the absolute value of the output is taken, a chain of
pulses results so that a failing LLA will signal its graceful degradation.

VLSI implementations of LLAs are described in the paper, with an example
device submitted to MOSIS, and due back in January 1990.  We are aware of
the work of Alspector et. al. and Graf et. al., which is specific to
neural architectures.  Our work is more general in that it arises from
a logic with both algebraic and logical semantics, lending the dual
semantics (and its generality) to the resulting device.

LLAs can also be integrated with the receptor circuits of Mead, leading
to a design project here for a single circuit that emulates the first
several levels of the visual system, not simply the retina.  This is
almost necessary because I can put over 2,000 processors on a single
chip, but haven't the input pins to drive them!  Thus, a chip that
uses fewer processors with the majority of inputs generated on chip is
quite attractive -- especially since even with faults I'll still get
a valid result from the computational part of the device.

Sincerely,

Jonathan Wayne Mills
Assistant Professor
Computer Science Department
Indiana University
Bloomington, Indiana 47405
(812) 331-8533

-----

From: risto at CS.UCLA.EDU (Risto Miikkulainen)

I did a brief analysis of the fault tolerance of distributed
representations. In short, as more units are removed from the
representation, the performance degrades linearly. This result is
documented in a paper I submitted to Cognitive Science a few days ago:

Risto Miikkulainen and Michael G. Dyer (1989). Natural Language
Processing with Modular Neural Networks and Distributed Lexicon.

Some preliminary results are mentioned in:

@InProceedings{miikkulainen:cmss,
  author = 	"Risto Miikkulainen and Michael G. Dyer",
  title = 	"Encoding Input/Output Representations in Connectionist
Cognitive Systems",
  booktitle = 	"Proceedings of the 1988 Connectionist Models Summer School",
  year = 	"1989",
  editor = 	"David S. Touretzky and Geoffrey E. Hinton and Terrence
J. Sejnowski",
  publisher = 	 KAUF,
  address = 	 KAUF-ADDR,
}

-----

"Implementation of Fault Tolerant Control Algorithms Using Neural
Networks", systematix, Inc., Report Number 4007-1000-08-89, August 1989.

-----

From: kddlab!tokyo-gas.co.jp!hajime at uunet.uu.net

>            " A study of high reliable systems
>        against electric noises and element failures "
> 
>         -- Apllication of neural network systems -- 
> 
> ISNCR '89 means "International Symposium on Noise and Clutter Rejection
> in Radars and Image Processing in 1989".
> It was held in Kyoto, JAPAN from Nov.13 to Nov.17.

Hajime FURUSAWA 	JUNET: hajime at tokyo-gas.co.jp
Masayuki KADO		JUNET: kado at tokyo-gas.co.jp

Research & Development Institute
Tokyo Gas Co., Ltd.
1-16-25 Shibaura, Minato-Ku
Tokyo 105
JAPAN

-----

From: <MJ_CARTE at UNHH.BITNET>  Mike Carter

"Operational Fault Tolerance of CMAC Networks", NIPS-89, by Mikeael J.
Carter, Frank Rudolph, and Adam Nucci, University of New Hampshire

Mike Carter also says he has a non-technical overview of NN fault
tolerance which he wrote some time ago which contains references to
papers which have some association with fault tolerance (although only
1 of which had fault tolerance as its focus).

-----

From: Martin Emmerson <mde at ecs.southampton.ac.uk>

I am working on simulating faults
in neural-networks using a program running
on a Sun (Unix and C).

I am particularly interested in qualitative methods
for assessing performance of a network and also
faults that might occur in a real VLSI implementation.