Connectionists: Lifelong Machine Learning please

Mon Apr 17 10:51:14 EDT 2017

Dear Hava and others …

What is in a name?
Lifelong Learning Machines <= Lifelong Machine Learning <= Machine Lifelong Learning <= Learning to Learn

All of the above are concerned with the persistent and cumulative nature of learning with machines.  They are based on the hypothesis that more efficient (shorter training times, fewer training examples) and more effective (more accurate hypotheses) learning relies on an appropriate inductive bias, one source being prior knowledge from related tasks (or examples from the same task domain).   They should also be concerned with the consolidation of knowledge acquired through learning to support inductive bias, which forces one to look at the representation of learned knowledge.

I have been studying Lifelong Machine Learning since 1993.   The field has gone from having no name to several vintages.  This is an appeal for the community to stay with the title “Lifelong Machine Learning” unless there is some need to distinquish “Lifelong Learning Machines” as a separate discipline.

In 1995, Rich Caruana and I organized the first NIPS workshop on “Learning to Learn: Knowledge Consolidation
and Transfer in Inductive Systems". See http://plato.acadiau.ca/courses/comp/dsilver/NIPS95_LTL/transfer.workshop.1995.html
This workshop produced a seminal book by Sebastian Thrun that solidified the title “Learning to Learn” or L2L. See http://robots.stanford.edu/papers/thrun.book3.html

Over the next decade myself and several others started to use the term “Machine Lifelong Learning” or ML3.
Our lab created a ML3 contributor website that has fallen behind over the years (http://ml3.acadiau.ca/) being replaced by material on our current lab website http://mlrl.acadiau.ca/ and by ResearchGate websites such as - https://www.researchgate.net/profile/Daniel_Silver

The L2L and ML3 titles lasted well into the first decade of the 2000s and was used at the second NIPS workshop on the subject “Inductive Transfer : 10 Years Later”.  See http://socrates.acadiau.ca/courses/comp/dsilver/Share/2005Conf/NIPS2005_ITWS/Website/index.htm

Along the way Mark Ring has distinquished “Continual Learning” in the Reinforcement Learning paradigm as a process of learning ever more complicated skills by building on those skills already developed.  See
https://www.cs.utexas.edu/~ring/Diss/index.html  and his new company http://www.cogitai.com/

Around about 2010, Eric Eaton and others started to use the term “Lifelong Machine Learning” or LML which many people have come to like.   Please see Eric’s webpage for some of the work he has been involved in https://www.seas.upenn.edu/~eeaton/research.html

So given that we have the well-used term “Lifelong Machine Learning” and that the name has changed a few times already, I really do not cherish the community moving toward yet another permutation of the three words “Lifelong”, “Machine”, and “Leaning”, unless it is really a different research area … In which case, I would ask that we use a significantly different monicker.  I make my case for sticking with the title “Lifelong Machine Learning” with the list of its uses shown below my signature.

Note that a new research theme is starting to be used that brings together machine learning and knowledge representation to solve one of the Big AI problems.  The problem is how to the learn background knowledge so it can be used for reason and the new title is “Lifelong Machine Learning and Reasoning”.
Recently, I created a ResearchGate project which is gaining followers
https://www.researchgate.net/project/Lifelong-Machine-Learning-and-Reasoning

.. Danny

==========================
Daniel L. Silver
Professor and Acting Director, Jodrey School of Computer Science
Director, Acadia Institute for Data Analytics
Acadia University,
Office 314, Carnegie Hall,
Wolfville, Nova Scotia Canada B4P 2R6

t. (902) 585-1413
f. (902) 585-1067

acadiau.ca<http://www2.acadiau.ca/index.php>
Facebook<https://www.facebook.com/acadiauniversity>  Twitter<https://twitter.com/acadiau>  YouTube<https://www.youtube.com/user/AcadiaWebmaster>  LinkedIn<https://www.linkedin.com/company/acadia-university?trk=biz-companies-cym>  Flickr<https://www.flickr.com/photos/acadiauniversity/albums>
[cid:image001.png at 01D2B770.E8E99CD0]

In recent years, there has come to exist a wide variety of websites that are currently using “Lifelong Machine Learning” including those related to:

Books:
https://www.cs.uic.edu/~liub/lifelong-machine-learning.html

Papers:
Lifelong machine learning: a paradigm for continuous learning<http://link.springer.com/content/pdf/10.1007/s11704-016-6903-6.pdf>
http://www.aaai.org/ocs/index.php/SSS/SSS13/paper/viewFile/5802/5977
http://dl.acm.org/citation.cfm?id=2433459
https://pdfs.semanticscholar.org/fb24/b6917eb42ccbf354371ee9565a3014b51e7c.pdf
https://cs.byu.edu/colloquium/sentiment-analysis-and-lifelong-machine-learning
https://scholar.google.com/citations?user=Z_vWXgsAAAAJ&hl=en

Popular Press Articles:
https://www.weforum.org/agenda/2017/01/lifelong-machine-learning/
http://www.rollproject.org/lifelong-machine-learning-systems-optimisation/

Videos:
https://www.youtube.com/watch?v=wc2xn4g1-uU

Courses:
https://www.cs.uic.edu/~liub/lifelong-learning.html

Tutorials and Workshops:
https://www.cs.uic.edu/~liub/IJCAI15-tutorial.html
https://www.seas.upenn.edu/~eeaton/AAAI-SSS13-LML/
http://repository.ust.hk/ir/Record/1783.1-73755
https://bigdata.cs.dal.ca/news/2014-06-09-000000/seminar-lifelong-machine-learning-and-reasoning

Research Websites:
http://mlrl.acadiau.ca/
https://www.cs.uic.edu/~liub/lifelong-learning.html
https://www.seas.upenn.edu/~eeaton/research.html
https://jaimefernandezdcu.wordpress.com/2016/10/24/lml/

++++++++   +++++++++  ++++++++++

From: Connectionists <connectionists-bounces at mailman.srv.cs.cmu.edu> on behalf of Hava Siegelmann <hava.siegelmann at gmail.com>
Date: Sunday, April 16, 2017 at 1:57 PM
To: "connectionists at mailman.srv.cs.cmu.edu" <connectionists at mailman.srv.cs.cmu.edu>
Subject: Re: Connectionists: Lifelong Learning Machines - Call for Grants

Dear Connectionists, it was a typo, Lifelong Learning Machine is NOW AVAILABLE  - please read, get together and apply.
We have the chance to start a new chapter of AI.

Hava

On Fri, Apr 14, 2017 at 5:41 PM, Hava Siegelmann <hava.siegelmann at gmail.com<mailto:hava.siegelmann at gmail.com>> wrote:
Dear Friends

Lifelong Learning Machines (L2M) call for proposals (or in DARPA lingo  BAA (Broad agency announcement)) is not available online from the DARPA portal

Note that there is also a link for teaming to enable create small groups is you are looking for collaborators.

Note that DARPA programs are once in a lifetime rather than NSF/NIH with repeating ideas. So start reading and prepare your applications on time.

All the best and much luck -

Hava Siegelmann

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20170417/2d3878f6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 7485 bytes
Desc: image001.png
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20170417/2d3878f6/attachment.png>