postdoc or visitor position and new papers available

[Eric B. Baum]

I am seeking applicants for a Post-doc or Visiting Scientist
position in my group at the NEC Research Institute in 
Princeton NJ. USA
The position is for 1 year.

To apply please send cv, cover letter and list of references to: 
Eric Baum, NEC Research Institute, 4 Independence Way, Princeton NJ 08540,USA
     or PREFERABLY by internet to eric at research.nj.nec.com


Our research is focused on artificial economies of agents
that reinforcement learn. Two new papers (and an extended
abstract of one of these) are available on my web page
http://www.neci.nj.nec.com:80/homepages/eric/eric.html

The abstracts of these papers are appended below.
-------------------------------------------------------------------
-------------------------------------------------------------------


  "Manifesto for an Evolutionary Economics of Intelligence"

( PostScript file 58 pages) Draft last modified July, 1998.
to appear in "Neural Networks and Machine Learning" Editor C. M. Bishop,
Springer-Verlag (1998).

We address the problem of reinforcement learning in ultra-complex
environments. Such environments will require a modular approach. The
modules must become rational in the sense that they learn to solve a piece 
of the problem. We describe how to use economic principles to assign credit 
and ensure that a collection of rational agents will collaborate on 
reinforcement learning. We also catalog several catastrophic failure modes 
that can be expected in distributed learning systems, and empirically have 
occurred in biological evolution, real economies, and artificial 
intelligence programs, when an appropriate economic structure is not 
enforced. 

We conjecture that such evolutionary economies can allow learning in 
feasible time scales, starting from a collection of agents which have 
little knowledge and hence are irrational, by dividing and conquering 
complex problems. We support this with two implementations of learning 
models based on these principles. The first of these systems has 
empirically learned to solve Blocks World problems involving arbitrary 
numbers of blocks. The second has demonstrated meta-learning-- it learns 
better ways of creating new agents, modifying its own learning algorithm 
to escape from local optima trapping competing approaches. We describe 
how economic models can naturally address problems at the meta-level, 
meta-learning and meta-computation, that are necessary for high 
intelligence; discuss the evolutionary origins and nature of biological
intelligence; and compare, analyze, contrast, and report experiments on
competing techniques including hillclimbing, genetic algorithms, genetic
programming, and temporal difference learning. 

----------------------------------------------------------
  "Toward Code Evolution by Artificial Economies"

E. B. Baum and Igor Durdanovic
( PostScript file 53 pages) Draft last modified July 8, 1998.
( PostScript file 9 pages) Extended Abstract,July 8, 1998.

We have begun exploring code evolution by artificial economies. We
implemented a reinforcement learning machine called Hayek2 consisting of
agents, written in a machine language inspired by Ray's Tierra, that 
interact economically. Hayek2 succeeds in evolving code to solve Blocks 
World problems, and has been more effective at this than our hillclimbing 
program and our genetic program. Our hillclimber and our genetic program, 
in turn, both performed creditably, learning solutions as strong 
as a simple search program that utilizes substantial hand-coded domain 
knowledge. We made some efforts to optimize our hillclimbing program and 
it has features that may be of independent interest. Our genetic program 
exhibited strong gains from crossover compared to a version utilizing 
other macro-mutations. The relative strength of crossover and 
macro-mutations is a hotly debated issue within the GP community, 
and ours is the first unequivocal demonstration of which we are aware
where crossover is much better than ``headless chicken mutation''. 

We have demonstrated meta-learning: Hayek2 succeeds in discovering new
meta-level agents that improve its performance, getting it out of 
plateaus in which it has otherwise gotten stuck. Hayek2's performance 
benefitted from improvements in the algorithm deciding how meta-agents 
gave capital to their offspring, from improvements in how creation of 
intellectal property is rewarded, from improvements in how meta-agents 
are paid by their offspring, from assessing a rent for computational 
time that was proportional to total demand, and from improvements in 
the language, including strong typing to bias the search for useful 
agents and expanding the representational power of the
meta-instructions using pattern based instructions. 


-------------------------------------
Eric Baum
NEC Research Institute, 4 Independence Way, Princeton NJ 08540
PHONE:(609) 951-2712, FAX:(609) 951-2482, Inet:eric at research.nj.nec.com
http://www.neci.nj.nec.com:80/homepages/eric/eric.html