Connectionists: Yang and Piantodosi’s PNAS language system, semantics, and scene understanding

[Gary Marcus]

– agree with Steve this is an interesting paper, and replicating it with a neural net would be interesting; cc’ing Steve Piantosi. 
— why not use a Transformer, though?
- it is however importantly missing semantics. (Steve P. tells me there is some related work that is worth looking into). Y&P speaks to an old tradition of formal language work by Gold and others that is quite popular but IMHO misguided, because it focuses purely on syntax rather than semantics.  Gold’s work definitely motivates learnability but I have never taken it to seriously as a real model of language
- doing what Y&P try to do with a rich artificial language that is focused around syntax-semantic mappings could be very interesting
- on a somewhat but not entirely analogous note, i think that  the next step in vision is really scene understanding. We have techniques for doing object labeling reasonably well, but still struggle wit parts and wholes are important, and with relations more generally, which is to say we need the semantics of scenes. is the chair on the floor, or floating in the air? is it supporting the pillow? etc. is the hand a part of the body? is the glove a part of the body? etc

Best,
Gary



> On Jun 13, 2022, at 05:18, jose at rubic.rutgers.edu wrote:
> 
> Again, I think a relevant project here  would be to attempt to replicate with DL-rnn, Yang and Piatiadosi's PNAS language learning system--which is a completely symbolic-- and very general over the Chomsky-Miller grammer classes.   Let me know, happy to collaborate on something like this.
> 
> Best
> 
> Steve
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20220613/cf6086cb/attachment.html>