Connectionists: Internship - Unsupervised learning of optic flow with spiking neural networks

[Adrien Fois]

Internship - Unsupervised learning of optic flow with spiking neural networks 







Supervisor : Bernard Girau and Adrien Fois 

Lab and team : LORIA (FRANCE), BISCUIT 

Contacts : [ mailto:bernard.girau at loria.fr | bernard.girau at loria.fr ] , [ mailto:adrien.fois at loria.fr | adrien.fois at loria.fr ] 


Start : Position open until filled 

Duration : 5-6 months 







Motivation and context 




Spiking neurons are considered the third generation of artificial neural models. These neural models take bio-mimicry a step further than their predecessors by communicating - in the manner of biological neurons - with spikes produced in time. A new dimension - the temporal dimension - thus allows information to be transmitted and processed on the fly, asynchronously. 




To take full advantage of the computing power and the very low energy consumption induced, these spiking neuron models can be directly emulated. This is what Intel and IBM have done with their Loihi neuromorphic processors and TrueNorth, respectively. Loihi2 integrates one million impulse neurons and 120 million programmable synapses. 




In the same bio-inspired vein, event-based cameras are gaining popularity. Event-based cameras such as DVS (Dynamic Vision Sensor) work analogously to the retina by transmitting information as a spike only when a local change in brightness - at the pixel level - is detected. This asynchronous processing of visual information brings great advantages: 1) a sampling speed nearly a million times faster than standard cameras, 2) a latency of one microsecond and 3) a dynamic range of 130 decibels (standard cameras have only 60 dB). All this with significantly lower power consumption than standard cameras. 




When an organism equipped with a visual system is in movement in its environment, or observes an object in movement while remaining static, it perceives a relative movement between itself and its environment. This motion appears to it in the form of spatio-temporal patterns, called optical flow. Estimating the optical flow is an essential task for the organism. This information allows it to better estimate its own movement and thus to better navigate in its environment. These problems are also transposable in the field of autonomous robotics and drones. 




This internship, which will last at least 5 months, is at the crossroads of these different fields. 







Goals and Objectives 




The goal is to use an event-driven camera and to process its data with a spiking neural network equipped with unsupervised learning rules. The intended application is optical flow estimation. 




This will include: 

- to carry out a bibliographical study on the methods of unsupervised optical flow estimation with impulse neural networks 

- to adapt a learning rule of the STDP type, developed within the team, to the targeted application 

- to integrate this rule into impulse neural networks 

- propose adaptations compatible with a hardware implementation on a neuromorphic processor 

- implement and test the architecture with Tensorflow 







A background in computer science (with a foundation in artificial intelligence) or computational neuroscience is expected, as well as a strong foundation in programming. 




The internship will take place in France in the Loria laboratory, in the Biscuit team. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20211118/95c5dd3b/attachment.html>