Connectionists: developmental robotics

[james bower]

> 
> 
> As far as language development is concerned, I would even say that most work I know assumes advanced prior computational structures for things like unification and recursion,

Sorry again to probably be revealing my ignorance on current practices in a field, but, isn’t the "prior computational structure", in this case, the actual structure of cerebral cortex itself.  Just like the structure of music, the structure of language, it seems to me, is very likely to deeply reflect the core structure of cerebral cortical networks, first and foremost.

A few years ago a friend of mine Jaron Lanier and I were talking about possible influences of the computational structure of the olfactory system on language.  What was an off the cuff conversation ended up as a chapter in his book “you are not a gadget” - 

How olfaction might have played a crucial role in the evolution of human language

Jaron didn’t get my sense completely right, but the general idea is there.

As I have mentioned before, I believe that a deeper understanding of the computational requirements for the olfactory system and how those requirements may be met by the core structure of cerebral cortical networks (which has nothing to do with columns), at least represents a different approach to understanding systems like language (and vision).

Jim



> within hybrid neural architectures [12] or based on older more classical computational architectures now used in concert with statistical inference for studying the development of grammatical structures in artificial agents [13, 14, 15]. Not everyone though pre-supposes those high-level symbolic compositional capabilities (some study their very formation out of neural architectures [18]), but in that case they still often consider as the central object of study the role of constraints, such as for example the role of embodiment and social cues to guide a statistical learner to pick regularities in language acquisition [16, 17]. 
> 
> Best,
> Pierre-Yves
> 
> 
> [1] Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints
> Oudeyer P-Y., Baranes A., Kaplan F. (2013)
> in Intrinsically Motivated Learning in Natural and Artificial Systems, eds. Baldassarre G. and Mirolli M., Springer.
> http://www.pyoudeyer.com/OudeyerBaranesKaplan13.pdf
> 
> [2] Information Seeking, Curiosity and Attention: Computational and Neural Mechanisms 
> Gottlieb, J., Oudeyer, P-Y., Lopes, M., Baranes, A. (2013) 
> Trends in Cognitive Science, 17(11), pp. 585-596.
> http://www.pyoudeyer.com/TICSCuriosity2013.pdf
> 
> [3] Maturational constraints for motor learning in high-dimensions : the case of biped walking
> Lapeyre, M., Ly, O., Oudeyer, P-Y. (2011)
> Proceedings of IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS 2011), Bled, Slovenia.
> http://hal.archives-ouvertes.fr/docs/00/64/93/33/PDF/Humanoid.pdf
> 
> [4] Poppy Humanoid Platform: Experimental Evaluation of the Role of a Bio-inspired Thigh Shape,
> Matthieu Lapeyre, Pierre Rouanet, Pierre-Yves Oudeyer, 
> Humanoids 2013.
> http://hal.inria.fr/docs/00/86/11/10/PDF/humanoid2013.pdf
> 
> [5] Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
> Adrien Baranes <logo_url.jpg>; Pierre-Yves Oudeyer <logo_url.jpg>
> Robotics and Autonomous Systems, Elsevier, 2013, 61 (1), pp. 69-73
> http://hal.inria.fr/docs/00/78/84/40/PDF/RAS-SAGG-RIAC-2012.pdf
> 
> [6] Active Choice of Teachers, Learning Strategies and Goals for a Socially Guided Intrinsic Motivation Learner
> Sao Mai Nguyen <logo_url.jpg>; Pierre-Yves Oudeyer <logo_url.jpg>
> Paladyn Journal of Bejavioral Robotics, Springer, 2012, 3 (3), pp. 136-146
> http://hal.inria.fr/docs/00/93/69/32/PDF/Nguyen_2012Paladyn.pdf
> 
> [7] Yasunori Yamada, Yasuo Kuniyoshi: 
> Embodiment guides motor and spinal circuit development in vertebrate embryo and fetus,
> International Conference on Development and Learning (ICDL2012)/ EpiRob2012, 2012.
> http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6400578
> 
> [8] J. Law, P. Shaw, K. Earland, M. Sheldon, M. Lee (2014) A psychology based approach for longitudinal development in cognitive robotics. Frontiers in Neurorobotics 8 (1) pp. 1-19.
> http://www.frontiersin.org/Neurorobotics/10.3389/fnbot.2014.00001/abstract
> 
> [9] A Simple Ontology of Manipulation Actions Based on Hand-Object Relations
> in: Autonomous Mental Development, IEEE Transactions on, Issue Date: June 2013, Written by: Worgotter, F.; Aksoy, E.E.; Kruger, N.; Piater, J.; Ude, A.; Tamosiunaite, M.
> https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6493410
> 
> [10] F. Guerin, N. Krüger and D. Kraft
> “A survey of the ontogeny of tool use: from sensorimotor experience to planning”
> IEEE Trans. Autonom. Mental Develop., vol. 5, no. 1, pp. 18-45, 2013
> https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6248675
> 
> [11] Katerina Pastra and Yiannis Aloimonos (2012), “The Minimalist Grammar of Action”, Philosophical Transactions of the Royal Society B, 367(1585):103.
> http://www.csri.gr/files/csri/docs/pastra/philoTrans-pastra.pdf
> 
> [12] Hinaut X, Dominey PF (2013) Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing Plos One 8(2): e52946
> http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0052946
> 
> [13] Siskind https://engineering.purdue.edu/~qobi/
> 
> [14] Beuls K, Steels L (2013) Agent-Based Models of Strategies for the Emergence and Evolution of Grammatical Agreement. PLoS ONE 8(3): e58960. doi:10.1371/journal.pone.0058960
> http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0058960
> 
> [15] Daniel Hewlett, Thomas Walsh, Paul Cohen. 2011. Teaching and Executing Verb Phrases .Development and Learning (ICDL), 2011 IEEE International Conference on 2, 1-6. 
> http://w3.sista.arizona.edu/~cohen/Publications/papers/Hewlett-2011.pdf
> 
> [16] Attentional constraints and statistics in toddlers' word learning
> SH Suanda, SB Foster, LB Smith, C Yu - Development and Learning and Epigenetic Robotics ( …, 2013
> http://www.indiana.edu/~dll/papers/SFSY_2013_ICDL.pdf
> 
> [17] The role of embodied intention in early lexical acquisition
> C Yu, DH Ballard, RN Aslin - Cognitive science, 2005
> 
> [18]  M. Maniadakisa, P. Trahaniasa, J. Tani: “Self-organizing high-order cognitive functions in artificial agents: implications for possible prefrontal cortex mechanisms”, Neural Networks, Vol33, pp.76-87, 2012.
> http://neurorobot.kaist.ac.kr/pdf_files/Maniadakis-NN2012.pdf
> 
> 
> On 11 Feb 2014, at 15:15, Gary Marcus <gary.marcus at nyu.edu> wrote:
> 
>> Pierre (and others)
>> 
>> Thanks for all the links. I am a fan, in principle, but much of the developmental robotics work I have seen sticks rigidly to a fairly “blank-slate” perspective, perhaps needlessly so. I’d especially welcome references to work in which researchers have given robots a significant head start, so that the learning that takes place has a strong starting point. Has anyone, for example, tried to build a robot that starts with the cognitive capacities of a two-year-old (rather than a newborn), and goes from there? Or taken seriously the nativist arguments of Chomsky, Pinker, and Spelke and tried to build a robot that is innately endowed with concepts like “person”, “object”, “set”, and “place”?
>> 
>> Best,
>> Gary
>> 
>> On Feb 11, 2014, at 6:58 AM, Pierre-Yves Oudeyer <pierre-yves.oudeyer at inria.fr> wrote:
>> 
>>> 
>>> Hi,
>>> 
>>> the view put forward by Gart strongly resonates with the approach that has been taken in the developmental robotics community in the last 10 years.
>>> I like to explain developmental robotics as the study of developmental constraints and architectures which guide learning mechanisms so as to allow actual lifelong acquisition and adaptation of skills in the large high-dimensional real world with severely limited time and space resources.
>>> Such an approach considers centrally the No Free Lunch idea, and indeed tries to identifies and understand specific families of guiding mechanisms that allow corresponding families of learners to acquire families of skills in some families of environments.
>>> For pointers, see: http://en.wikipedia.org/wiki/Developmental_robotics
>>> 
>>> Thus, in a way, while lifelong learning and adaptation is a key object of study, most work is not so much about elaborating new models of learning mechanisms, but about studying what (often changing) properties of the inner and outer environment allow to canalise them.
>>> Examples of such mechanisms include body and neural maturation, active learning (selection of action that provide informative and useful data), emotional and motivational systems, cognitive biases for inference, self-organisation or socio-cultural scaffolding, and their interactions.
>>> 
>>> This body of work is unfortunately not yet well connected (except a few exceptions) with the connectionnist community, but I am convinced more mutual exchange would be valuable.
>>> 
>>> For those interested, we have a dedicated:
>>> - journal: IEEE TAMD https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4563672
>>> - conference: IEEE ICDL-Epirob
>>> - newsletter: IEEE CIS AMD, latest issue: http://www.cse.msu.edu/amdtc/amdnl/AMDNL-V10-N2.pdf
>>> 
>>> Best regards,
>>> Pierre-Yves Oudeyer
>>> http://www.pyoudeyer.com
>>> https://flowers.inria.fr
>>> 
>>> On 10 Feb 2014, at 17:38, Gary Marcus <gary.marcus at nyu.edu> wrote:
>>> 
>>>> Juergen and others,
>>>> 
>>>> I am with John on his two basic concerns, and think that your appeal to computational universality is a red herring; I cc the entire group because I think that these issues lay at the center of why many of the hardest problems in AI and neuroscience continue to lay outside of reach, despite in-principle proofs about computational universality. 
>>>> 
>>>> John’s basic points, which I have also made before (e.g. in my books The Algebraic Mind and The Birth of the Mind and in my periodic New Yorker posts) are two
>>>> 
>>>> a. It is unrealistic to expect that hierarchies of pattern recognizers will suffice for the full range of cognitive problems that humans (and strong AI systems) face. Deep learning, to take one example, excels at classification, but has thus far had relatively little to contribute to inference or natural language understanding.  Socher et al’s impressive CVG work, for instance, is parasitic on a traditional (symbolic) parser, not a soup-to-nuts neural net induced from input. 
>>>> 
>>>> b. it is unrealistic to expect that all the relevant information can be extracted by any general purpose learning device.
>>>> 
>>>> Yes, you can reliably map any arbitrary input-output relation onto a multilayer perceptron or recurrent net, but only if you know the complete input-output mapping in advance. Alas, you can’t be guaranteed to do that in general given arbitrary subsets of the complete space; in the real world, learners see subsets of possible data and have to make guesses about what the rest will be like. Wolpert’s No Free Lunch work is instructive here (and also in line with how cognitive scientists like Chomsky, Pinker, and myself have thought about the problem). For any problem, I presume that there exists an appropriately-configured net, but there is no guarantee that in the real world you are going to be able to correctly induce the right system via general-purpose learning algorithm given a finite amount of data, with a finite amount of training. Empirically, neural nets of roughly the form you are discussing have worked fine for some problems (e.g. backgammon) but been no match for their symbolic competitors in other domains (chess) and worked only as an adjunct rather than an central ingredient in still others (parsing, question-answering a la Watson, etc); in other domains, like planning and common-sense reasoning, there has been essentially no serious work at all.
>>>> 
>>>> My own take, informed by evolutionary and developmental biology, is that no single general purpose architecture will ever be a match for the endproduct of a billion years of evolution, which includes, I suspect, a significant amount of customized architecture that need not be induced anew in each generation.  We learn as well as we do precisely because evolution has preceded us, and endowed us with custom tools for learning in different domains. Until the field of neural nets more seriously engages in understanding what the contribution from evolution to neural wetware might be, I will remain pessimistic about the field’s prospects.
>>>> 
>>>> Best,
>>>> Gary Marcus
>>>> 
>>>> Professor of Psychology
>>>> New York University
>>>> Visiting Cognitive Scientist
>>>> Allen Institute for Brain Science
>>>> Allen Institute for Artiificial Intelligence
>>>> co-edited book coming late 2014:
>>>> The Future of the Brain: Essays By The World’s Leading Neuroscientists
>>>> http://garymarcus.com/
>>>> 
>>>> On Feb 10, 2014, at 10:26 AM, Juergen Schmidhuber <juergen at idsia.ch> wrote:
>>>> 
>>>>> John,
>>>>> 
>>>>> perhaps your view is a bit too pessimistic. Note that a single RNN already is a general computer. In principle, dynamic RNNs can map arbitrary observation sequences to arbitrary computable sequences of motoric actions and internal attention-directing operations, e.g., to process cluttered scenes, or to implement development (the examples you mentioned). From my point of view, the main question is how to exploit this universal potential through learning. A stack of dynamic RNN can sometimes facilitate this. What it learns can later be collapsed into a single RNN [3].
>>>>> 
>>>>> Juergen
>>>>> 
>>>>> http://www.idsia.ch/~juergen/whatsnew.html
>>>>> 
>>>>> 
>>>>> 
>>>>> On Feb 7, 2014, at 12:54 AM, Juyang Weng <weng at cse.msu.edu> wrote:
>>>>> 
>>>>>> Juergen:
>>>>>> 
>>>>>> You wrote: A stack of recurrent NN.  But it is a wrong architecture as far as the brain is concerned.
>>>>>> 
>>>>>> Although my joint work with Narendra Ahuja and Thomas S. Huang at UIUC was probably the first
>>>>>> learning network that used the deep Learning idea for learning from clutter scenes (Cresceptron ICCV 1992 and IJCV 1997),
>>>>>> I gave up this static deep learning idea later after we considered the Principle 1: Development.
>>>>>> 
>>>>>> The deep learning architecture is wrong for the brain.  It is too restricted, static in architecture, and cannot learn directly from cluttered scenes required by Principle 1.  The brain is not a cascade of recurrent NN.
>>>>>> 
>>>>>> I quote from Antonio Damasio "Decartes' Error": p. 93: "But intermediate communications occurs also via large subcortical nuclei such as those in the thalamas and basal ganglia, and via small nulei such as those in the brain stem."
>>>>>> 
>>>>>> Of course, the cerebral pathways themselves are not a stack of recurrent NN either.
>>>>>> 
>>>>>> There are many fundamental reasons for that.  I give only one here base on our DN brain model:  Looking at a human, the brain must dynamically attend the tip of the nose, the entire nose, the face, or the entire human body on the fly.  For example, when the network attend the nose, the entire human body becomes the background!  Without a brain network that has both shallow and deep connections (unlike your stack of recurrent NN), your network is only for recognizing a set of static patterns in a clean background.  This is still an overworked pattern recognition problem, not a vision problem.
>>>>>> 
>>>>>> -John
>>>>>> 
>>>>>> On 2/6/14 7:24 AM, Schmidhuber Juergen wrote:
>>>>>>> Deep Learning in Artificial Neural Networks (NN) is about credit assignment across many subsequent computational stages, in deep or recurrent NN.
>>>>>>> 
>>>>>>> A popluar Deep Learning NN is the Deep Belief Network (2006) [1,2].  A stack of feedforward NN (FNN) is pre-trained in unsupervised fashion. This can facilitate subsequent supervised learning.
>>>>>>> 
>>>>>>> Let me re-advertise a much older, very similar, but more general, working Deep Learner of 1991. It can deal with temporal sequences: the Neural Hierarchical Temporal Memory or Neural History Compressor [3]. A stack of recurrent NN (RNN) is pre-trained in unsupervised fashion. This can greatly facilitate subsequent supervised learning.
>>>>>>> 
>>>>>>> The RNN stack is more general in the sense that it uses sequence-processing RNN instead of FNN with unchanging inputs. In the early 1990s, the system was able to learn many previously unlearnable Deep Learning tasks, one of them requiring credit assignment across 1200 successive computational stages [4].
>>>>>>> 
>>>>>>> Related developments: In the 1990s there was a trend from partially unsupervised [3] to fully supervised recurrent Deep Learners [5]. In recent years, there has been a similar trend from partially unsupervised to fully supervised systems. For example, several recent competition-winning and benchmark record-setting systems use supervised LSTM RNN stacks [6-9].
>>>>>>> 
>>>>>>> 
>>>>>>> References:
>>>>>>> 
>>>>>>> [1] G. E. Hinton, R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, Vol. 313. no. 5786, pp. 504 - 507, 2006. http://www.cs.toronto.edu/~hinton/science.pdf
>>>>>>> 
>>>>>>> [2] G. W. Cottrell. New Life for Neural Networks. Science, Vol. 313. no. 5786, pp. 454-455, 2006. http://www.academia.edu/155897/Cottrell_Garrison_W._2006_New_life_for_neural_networks
>>>>>>> 
>>>>>>> [3] J. Schmidhuber. Learning complex, extended sequences using the principle of history compression, Neural Computation, 4(2):234-242, 1992. (Based on TR FKI-148-91, 1991.)  ftp://ftp.idsia.ch/pub/juergen/chunker.pdf  Overview: http://www.idsia.ch/~juergen/firstdeeplearner.html
>>>>>>> 
>>>>>>> [4] J. Schmidhuber. Habilitation thesis, TUM, 1993. ftp://ftp.idsia.ch/pub/juergen/habilitation.pdf . Includes an experiment with credit assignment across 1200 subsequent computational stages for a Neural Hierarchical Temporal Memory or History Compressor or RNN stack with unsupervised pre-training [2] (try Google Translate in your mother tongue): http://www.idsia.ch/~juergen/habilitation/node114.html
>>>>>>> 
>>>>>>> [5] S. Hochreiter, J. Schmidhuber. Long Short-Term Memory. Neural Computation, 9(8):1735-1780, 1997. Based on TR FKI-207-95, 1995.  ftp://ftp.idsia.ch/pub/juergen/lstm.pdf . Lots of of follow-up work on LSTM under http://www.idsia.ch/~juergen/rnn.html
>>>>>>> 
>>>>>>> [6] S. Fernandez, A. Graves, J. Schmidhuber. Sequence labelling in structured domains with hierarchical recurrent neural networks. In Proc. IJCAI'07, p. 774-779, Hyderabad, India, 2007.  ftp://ftp.idsia.ch/pub/juergen/IJCAI07sequence.pdf
>>>>>>> 
>>>>>>> [7] A. Graves, J. Schmidhuber. Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. NIPS'22, p 545-552, Vancouver, MIT Press, 2009.  http://www.idsia.ch/~juergen/nips2009.pdf
>>>>>>> 
>>>>>>> [8] 2009: First very deep (and recurrent) learner to win international competitions with secret test sets: deep LSTM RNN (1995-) won three connected handwriting contests at ICDAR 2009 (French, Arabic, Farsi), performing simultaneous segmentation and recognition.  http://www.idsia.ch/~juergen/handwriting.html
>>>>>>> 
>>>>>>> [9] A. Graves, A. Mohamed, G. E. Hinton. Speech Recognition with Deep Recurrent Neural Networks. ICASSP 2013, Vancouver, 2013.   http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Juergen Schmidhuber
>>>>>>> http://www.idsia.ch/~juergen/whatsnew.html
>>>>>> 
>>>>>> -- 
>>>>>> --
>>>>>> Juyang (John) Weng, Professor
>>>>>> Department of Computer Science and Engineering
>>>>>> MSU Cognitive Science Program and MSU Neuroscience Program
>>>>>> 428 S Shaw Ln Rm 3115
>>>>>> Michigan State University
>>>>>> East Lansing, MI 48824 USA
>>>>>> Tel: 517-353-4388
>>>>>> Fax: 517-432-1061
>>>>>> Email: weng at cse.msu.edu
>>>>>> URL: http://www.cse.msu.edu/~weng/
>>>>>> ----------------------------------------------
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 

 

 

Dr. James M. Bower Ph.D.

Professor of Computational Neurobiology

Barshop Institute for Longevity and Aging Studies.

15355 Lambda Drive

University of Texas Health Science Center 

San Antonio, Texas  78245

 

Phone:  210 382 0553

Email: bower at uthscsa.edu

Web: http://www.bower-lab.org

twitter: superid101

linkedin: Jim Bower

 

CONFIDENTIAL NOTICE:

The contents of this email and any attachments to it may be privileged or contain privileged and confidential information. This information is only for the viewing or use of the intended recipient. If you have received this e-mail in error or are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of, or the taking of any action in reliance upon, any of the information contained in this e-mail, or

any of the attachments to this e-mail, is strictly prohibited and that this e-mail and all of the attachments to this e-mail, if any, must be

immediately returned to the sender or destroyed and, in either case, this e-mail and all attachments to this e-mail must be immediately deleted from your computer without making any copies hereof and any and all hard copies made must be destroyed. If you have received this e-mail in error, please notify the sender by e-mail immediately.

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20140212/366b096c/attachment.html>