Connectionists: How the brain works

[Hussein Abbass]

Janet is asking an important question. What a single equation means to me is that the model is connected and I can fit it in my working memory to understand it. I would suggest the following three principles as a minimum for a piece of work to leave the lab:

1.      The Minimum Knowledge Principle
2.      The Scalability Vs Minimalist Structure Principle
3.      The Fit-for-Purpose or Validity Principle

Let me explain them in more details.

1.      The Minimum Knowledge Principle

This principle can also be called, the high school principle (for scientists) or the 7-year-old (for journalists) principle.  If we would like people to accept a model, they need to understand it. If they don't, it does not really matter how fascinating it is, or even how right it is (whatever right means), they will always resist it. BP is successful because a high school student with high school math can understand it. A journalist message is successful when a 7-year-old can understand it. The higher the bar of the minimum knowledge required to understand a model is, the less likely the model will be widely accepted or adopted.

In summary, the bar for the minimum knowledge needed to understand and use the model should be on an appropriate level for multiple groups in different sub-disciplines to understand it.

2.      The Scalability Vs Minimalist Structure Principle

The classical Feed-forward NN is successful because we can teach it with 2 neurons and implement it with hundreds or more nodes without a problem (obviously I am not downgrading numerical problems here). We can reduce the structure to a very small size to explain it and a single human can comprehend it properly, and we can scale it up to a very large dimension for real world problems while maintaining the principles of the minimalist structure. This is evident in many cases around us. Let us ask, what are the most popular data mining models in industry? Feed forward NN, decision trees, K-mean cluster, etc? They all share the same features, you can explain each of them on one page, you can have larger versions, the model is still the same.

3.      The Fit-for-Purpose or Validity Principle

How does the brain work? It is indeed a great question to ask but it seems to me there is an equally important question to ask as well, what is the purpose of asking the question? If the purpose is simply to advance our knowledge of how the brain works, then we need to ask, how do we know that we truly advanced this knowledge? Instead of continuing my argument with a series of questions, I will cut it short. We need to define the purpose(s) and be in a position to make sensible judgment that the advances we do fulfil this purpose.

The only way we can have a system for an ant that is identical to an ant is to create a biological ant. If we do not, there is always a distance, a gap between what we have (call it model, knowledge, etc) and what the biological system is. A purpose needs to be defined to tell us if we accept the gap or not. Feed forward NN is acceptable for basic regression purposes, but not acceptable as a biologically sound explanation of how the brain works. In the former, the gap is minimal, while in the latter, the gap and distance from the purpose is huge.

More principles can be added, but the above ones, I see most critical.

Kind regards,
Hussein

-----Original Message-----
From: Connectionists [mailto:connectionists-bounces at mailman.srv.cs.cmu.edu] On Behalf Of Janet Wiles
Sent: Friday, 23 May 2014 7:01 AM
To: Yu Shan
Cc: connectionists at mailman.srv.cs.cmu.edu
Subject: Re: Connectionists: How the brain works

When does a model escape from a research lab? Or in other words, when do researchers beyond the in-group investigate, test, or extend a model?

I have asked many colleagues this question over the years. Well-written papers help, open source code helps, tutorials help. But the most critical feature seems to be that it can be communicated in a single equation. Think about backprop, reinforcement learning, Bayes theorem.

Janet Wiles
Professor of  Complex and Intelligent Systems,
School of Information Technology and Electrical Engineering
The University of Queensland


-----Original Message-----
From: Connectionists [mailto:connectionists-bounces at mailman.srv.cs.cmu.edu] On Behalf Of Yu Shan
Sent: Friday, 23 May 2014 7:37 AM
To: Juyang Weng
Cc: connectionists at mailman.srv.cs.cmu.edu
Subject: Re: Connectionists: How the brain works

> Suppose that one gave all in this connectionists list a largely
> correct model about how the brain works, few on this list would be
> able to understand it let alone agree with it!
>

Let's look at a recent example. Nikolic proposed his theory
(http://www.danko-nikolic.com/practopoiesis/) about how the brain works a few weeks ago to the Connectionists. Upon finishing reading this paper, I was quite exited. The theory is elegantly simple and yet has great explanatory power. It is also consistent with what we know about evolution as well as the brain's organization and development.
Of course, we don't know yet if it is a "largely correct model about how the brain works". But, to my opinion, it has a great potential.
Actually I am thinking how to implement those ideas in my own future research.

However, the author's efforts of introducing this work to the Connectionists received little attention. Connectionists reach 5000+ people, who are probably the most interested and capable audience for such a topic. This makes the silence particularly intriguing. Of course, one possible reason is that lots of people here already studied this theory and deemed it irrelevant.

But a more likely reason, I think, is most people did not give it much thought. If that is the case, it raises an interesting question: what is the barrier that a theory of how the brain works need to overcome in order to be treated seriously? In other words, what do we really want to know?

Shan Yu, Ph.D
Brainnetome Center and National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing 100190, P. R. China http://www.brainnetome.org/en/shanyu

________________________________

THE UNIVERSITY OF NEW SOUTH WALES
UNSW CANBERRA AT THE AUSTRALIAN DEFENCE FORCE ACADEMY
PO Box 7916, CANBERRA BC 2610, Australia
Web: http://unsw.adfa.edu.au

CRICOS Provider no. 00100G

This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender and are not necessarily the views of UNSW.