Connectionists: modularity vs. sparsity

[Lee Giles]

I believe Joshua has an interesting point. In our '97 paper in IEEE TSP
titled "A Delay Damage Model Selection Algorithm for NARX Neural
Networks," we found significant performance improvement with
memory pruning which is a special kind of imposed cost on connection
lengths.
It would be interesting to redo these experiments to see if modularity
as defined by Clune, et al. emerges. I speculate it would.

(Paper attached for convenience)

On 3/4/13 11:53 AM, Joshua C. Bongard wrote:
> A lot of discussion here lately about the evolution of modularity,
> sparked by the recent paper
>
> http://arxiv.org/abs/1207.2743
>
> In it, the authors evolve ANNs to perform a specific task while placing
> a penalty on total wiring length.
>
> There were a lot of responses here along the lines of "it's not
> surprising
> that they got modularity because they placed a cost on connection
> length."
>
> However, what I found interesting about this paper is that they got
> modularity rather than sparsity. It is important to keep in mind that
> these two properties of networks are not the same thing.
>
> So, if I were to use Clune et al.'s approach to evolve ANNs for another
> task, would I get modularity or sparsity?
>
> Food for thought.
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IEEETSP-1997-delay-damage.pdf
Type: application/download
Size: 297974 bytes
Desc: not available
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/20130306/4dd9bdbb/attachment-0001.bin>