connectionist step function manipulator

[John.Hampshire@SPEECH2.CS.CMU.EDU]

If you want to time-shift and amplitude-scale
a step function, I think it's easiest to build
a simple algorithm to do this.  On the assumption
that this approach is undesirable for some reason,
I'd offer the following:

Look into classic linear systems/signal processing theory.
Linear systems theory focusses heavily on the step response
(and impulse response) of linear systems and how these
time-domain responses relate to frequency-domain
representations via Fourier transformation.
One way to get your desired network would start with
specifying the desired step response you want,
differentiating it in the time domain to obtain the
equivalent impulse response, then Fourier transforming
this result for the frequency domain equivalent.
If your desired step response involves a time
shifting and amplitude scaling, then these manipulations
constitute a pretty simple impulse response that transforms to
a phase-shift and amplitude scaling in the frequency domain.

There is a huge volume of information on the Fast Fourier
Transform (dating back to C. F. Gauss, but generally associated
with Cooley and Tukey [1965]).  Probably the best overview is
given by Oppenheim and Schafer in "Discrete Time Signal Processing"
(Prentice Hall, 1989) or the predecessor by the same authors and
publisher "Digital Signal Processing" (1975).  One can
view the various implementations of the FFT as neural networks
despite the fact that they are ultimately linear operators.
I imagine a some folks will dispute this equation, but if
adaptive phasing devices for antenna arrays are viewed as
neural nets, FFTs can be too.

Anyway, if you must use a "neural network", you could implement
your desired function in discrete time using an N-point
FFT of your input, a frequency-domain multiplication of the
transformed input and transformed impulse response, then
an N-point inverse FFT.  Oppenheim and Schafer (1989)
chapter 8 covers this concept pretty well (there's more
to it than this explanation).  If you wanted
to *learn* the desired time-shift and scaling factor
(instead of deriving it a-priori), I imagine you could
set up the FFT and inverse FFT structures and then use an
error function with backpropagation (through the UN-CHANGING
"connections" of the inverse  FFT stage of the network).
The error signal would backpropagate to alter the
coefficients of your frequency domain representation of
the impulse response.  Of course, those
frequency domain coefficients are *complex*, not pure real
(as are a number of the "connections" in the inverse FFT
structure), and I haven't really considered why such an
idea might not work, but what the heck.  Then there's
the question of *why* you'd want to learn the frequency
domain form of the impulse response if you can derive it
in closed form a-priori, but I'm sure there's a good
reason associated with your wanting to use a
connectionist architecture.

If all of this is old news to you, then sorry --- toss
it in the trash.  If it's new info then I hope it's of
some help.

Cheers,

John

P. S.  all of this is really just cramming traditional signal
processing into connectionist packaging.  I don't mean to
claim otherwise.