Connections Per Second

[Geoffrey Hinton]

I agree with previous messages that the problem with reporting "connections
per second" is that it could mean many different things:

1. CPS for just a forward pass  (2 FLOPS per connection)
2. CPS for forward & backward passes (usually 6 FLOPS per connection)
3. CPS for forward & backward passes on connections from input units (4 FLOPS)
4. CPS for forward & backward passes with online weight updates (8 or 6 FLOPS).

The obvious solution is to just report the megaflops, where the only
operations that are counted are the floating point operations in the inner
loops (i.e. we ignore any computations used to implement the sigmoid etc, but
we include the weight updating if we are using the online method, so we use
the numbers given above).  I expect that unscrupulous salespersons will still
try to cheat by counting operations like loads and stores and increments to
loop counters in the megaflop rating, but at least the researchers will have a
standard figure that can be used to compare machines and simulators.

Geoff