Connectionists: McKernel: A Library for Approximate Kernel Expansions in Log-linear Time.

["J. D. Curtó"]

Dear Colleagues and Friends,


We are pleased to announce the release (v2.2) of McKernel: A Library for Approximate Kernel Expansions in Log-linear Time (https://arxiv.org/pdf/1702.08159.pdf).


Abstract:
McKernel introduces a framework to use kernel approximates in the minibatch setting with Stochastic Gradient Descent (SGD) as an alternative to Deep Learning. Based on Random Kitchen Sinks [Rahimi and Recht 2007], we provide a C++ library for Large-scale Machine Learning. It contains a CPU optimized implementation of the algorithm in [Le et al. 2013], that allows the computation of approximated kernel expansions in log-linear time. The algorithm requires to compute the product of matrices Walsh Hadamard. A cache friendly Fast Walsh Hadamard that achieves compelling speed and outperforms current state-of-the-art methods has been developed. McKernel establishes the foundation of a new architecture of learning that allows to obtain large-scale non-linear classification combining lightning kernel expansions and a linear classifier. It travails in the mini-batch setting working analogously to Neural Networks. We show the validity of our method through extensive experiments on MNIST and FASHION MNIST [Xiao et al. 2017].


Code (by Curtó and Zarza):
https://github.com/curto2/mckernel


Please also check its follow-up work:


Doctor of Crosswise: Reducing Over-parametrization in Neural Networks (https://arxiv.org/pdf/1905.10324.pdf).


Abstract:
Dr. of Crosswise proposes a new architecture to reduce over-parametrization in Neural Networks. It introduces an operand for rapid computation in the framework of Deep Learning that leverages learned weights. The formalism is described in detail providing both an accurate elucidation of the mechanics and the theoretical implications.


Code (by Curtó and Zarza):
https://github.com/curto2/dr_of_crosswise


Best regards,
De Curtó.


--
J. D. Curtó
http://decurto.ch/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20200219/585abef2/attachment.html>