Connectionists: Postdocs in Riemannian Manifold Optimization, Deep Learning and Information Geometry at RIST, Romania

[Luigi Malagò]

[Our apologies for cross postings]

The Romanian Institute of Science and Technology (RIST) has an opening for
3 postdoc positions, in the context of the DeepRiemann project “Riemannian
Optimization Methods for Deep Learning”, funded by European structural
funds through the Competitiveness Operational Program (POC 2014-2020). The
appointments will be for 2 years, with possible extensions up to 4 years.

The DeepRiemann project aims at the design and analysis of novel training
algorithms for Neural Networks in Deep Learning, by applying notions of
Riemannian optimization and differential geometry. The task of the training
a Neural Network is studied by employing tools from Optimization over
Manifolds and Information Geometry, by casting the learning process to an
optimization problem defined over a statistical manifold, i.e., a set of
probability distributions. The project is highly interdisciplinary, with
competences spanning from Machine Learning to Optimization, Deep Learning,
Statistics, and Differential Geometry. The objectives of the project are
multiple and include both theoretical and applied research, together with
industrial activities oriented to transfer knowledge, from the institute to
a startup or spin-off of the research group.

The positions will be part of the new Machine Learning and Optimization
group, which will be performing research at the intersection of Machine
Learning, Stochastic Optimization, Deep Learning, and Optimization over
Manifolds, from the unifying perspective of Information Geometry. The group
is one of two newly-formed groups in Machine Learning at RIST, where about
20 new postdoctoral research associates and research software developers
will be hired in the next year.

The official job announcement can be seen here:
http://rist.ro/en/details/news/postdoc-positions-in-machine-
learning-optimization-deep-learning-and-information-geometry.html

# Jobs Description #

The three open positions will focus on different and overlapping aspects of
the project:

1) Optimization Algorithms over Statistical Manifolds with Applications to
Deep Learning

The postdoctoral researcher will conduct research on the design and
implementation of novel first- and second-order methods for the
optimization of functions defined over statistical models, such as
exponential families, the Gaussian distribution, and Markov Random Fields.
In particular the research will focus on the optimization problem
associated to the training of a Neural Network, which will be studied from
the perspective of Optimization over Manifolds with non-Euclidean
geometries. The postdoctoral candidate will develop optimization algorithms
which explicitly take into account the Riemannian and dual-affine Hessian
geometries of the search space, given by the Fisher information metric.
Such algorithms should be efficient, scalable, largely parallelizable, and
suitable for the large scale and big data settings.

Keywords: stochastic gradient descent, Riemannian manifolds, dually-flat
Hessian geometries, optimization over manifolds, second-order optimization
methods, large scale optimization.

2) Theory of Neural Networks

The postdoctoral researcher will contribute towards the definition of a
probabilistic and geometric framework for the study of deep Neural Networks
aimed at a better understanding of the working mechanisms behind the
success of Deep Learning. The analysis is focused towards a better
understanding of what enables Deep Learning systems to achieve remarkable
successes, and of its limits, in classification and predictive analysis,
with a particular focus on the learning task, in order to guide the
development of novel algorithms. The research will include the study of the
representational power of various Neural Networks topologies, by taking
into consideration the impact of hierarchical architectures with respect to
the expressiveness of the network; and the analysis of the number of local
minima and of saddle points, which appear during the training and which
strongly affect convergence of the optimization algorithms, depending on
the nature of the network.

Keywords: theory of neural networks, deep learning, learning
representations, expressive power of neural networks.

3) Information Geometry of dually-flat Hessian Manifolds

The postdoctoral researcher will focus on the characterization of first and
second-order Riemannian and affine geometries of statistical models, or
more in general Hessian manifolds, aimed at the study of optimization
methods over statistical manifolds. Neural Networks can be modeled as
statistical manifolds, or manifolds of Neural Networks, and the training of
the network can be studied as an optimization problem over a non-linear
manifold, where the weights of the network are the parameters of the model
and thus the variables of the function to be optimized. The purpose of the
analysis is to introduce alternative second-order approximations of a
function over a statistical manifold, depending on the choice of the
connection and thus how parallel transport is implemented, which are at the
basis of the design of second-order optimization methods.

Keywords: information geometry, geometry of statistical models, natural
gradient, dually-flat manifolds, Hessian geometries.

# Desired Qualifications #

- PhD in machine learning, theoretical computer science, stochastic
optimization, statistics, applied mathematics, including fields such as
manifold optimization, differential geometry, statistical mechanics, and
related fields
- Strong publication record
- Strong analytical skills, such as problem solving and logical thinking
- Enthusiasm to work in a multidisciplinary and international research
environment
- Good written and oral communication skills in English

Knowledge in machine learning is helpful, but not strictly mandatory.
Doctoral students close to the competition of their thesis will also be
considered.

The positions are to start as early as December 2016 or at any agreed later
date. Applications will be reviewed as they are received.

RIST offers competitive salaries and top-level working conditions. The net
salary for these positions will be around 2.000 euro per month, with a
possible increase up to 2.200 euro, based on new tax incentives for
research and development activities in Romania. Positions are endowed with
travel resources. The cost of living in Cluj is significantly lower than in
Western Europe or the USA (e.g., it is 1/3 of the cost of living in London,
UK).

# How to Apply #

In order to apply to this position, the candidate should send an email to
deepriemann.jobs at rist.ro, mentioning in the subject of the email
"DeepRiemann Postdoc Application". The email should include among the
attachments:

- A cover letter
- A complete CV with full list of publications
- A short research statement (max 3 pages), which describes your research
interests and explain why your skills, knowledge and experience makes you a
suitable candidate for one or more of the open positions
- The pdfs of 2 selected publications
- Name and contact information of up to 3 referees, which will be contacted
directly by the institute for a reference letter

Informal inquiries can be sent to Dr. Luigi Malagò <malago at rist.ro>,
Principal Investigator of the DeepRiemann project.

# About the Institute #

The Romanian Institute of Science and Technology is a non-governmental,
not-for-profit, independent research institute, founded in 2009, with the
purpose of offering scientists a place to conduct research in Romania with
top working conditions, comparable to those you can find in Western Europe.
RIST currently performs research on computational and experimental
neuroscience, computational intelligence, machine learning, and dynamical
systems. The institute is currently undergoing a phase of significant and
sustained growth supported by European and Romanian structural funds, with
plans for hiring around 20 new scientists and researchers in the next year.
The institute has close contacts with the Babeș-Bolyai University, which is
the largest university in Romania, and with the Technical University of
Cluj-Napoca.

The institute is located in Cluj-Napoca, in the heart of Transylvania,
which has been named by Lonely Planet as the top region to visit in 2016.
Cluj is a welcoming and innovative city, recently listed as one of the
major tech hubs (with a 9% growth per year for IT industry). Cluj has 12
universities, over 70k students every year, and an extremely vibrant
startup scene, which has recently named the Silicon Valley of Transylvania.

Cluj is Europe's friendliest city for foreigners, according to a study by
the UK Office of National Statistics, and Romania is no. 16 in the
Internations best expat destinations of 2016. About 1100 French citizens,
800 Italian citizens and 500 German citizens live in Cluj (source). The
city has an international airport, only 8km away from the city center, with
flights to more than 30 European destinations.

More information about Cluj:

https://vimeo.com/171864331
https://vimeo.com/166021527
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20161027/b13b1182/attachment.html>