À la Une

Soutenance de thèse Magda Gregorova


Mme Magda Gregorova soutiendra en anglais, en vue de l'obtention du grade de docteur ès sciences, mention informatique, sa thèse intitulée:

Sparse Learning for variable selection with structures and nonlinearities

Date: Lundi 12 novembre 2018 à 14h00

Lieu: CUI / Battelle bâtiment B, salle B3.08 (3ème étage)


Cette soutenance de thèse est précédée par un talk du Dr. Julien Mairal (à 11h00)



  • Prof. Stéphane Marchand-Maillet, département d'informatique, Université de Genève (co-director)
  • Dr. Alexandros Kalousis, département d'informatique, Université de Genève (co-director)
  • Dr. Julien Mairal, INRIA, Grenoble, France (external jury member)
  • Prof. Lorenzo Rosasco, University of Genova, Italy (external jury member)


In this thesis we discuss machine learning methods performing automated variable selection for learning sparse predictive models. There are multiple reasons for promoting sparsity in the predictive models. By relying on a limited set of input variables the models naturally counteract the overfitting problem ubiquitous in learning from finite sets of training points. Sparse models are cheaper to use for predictions, they usually require lower computational resources and by relying on smaller sets of inputs can possibly reduce costs for data collection and storage. Sparse models can also contribute to better understanding of the investigated phenomenons as they are easier to interpret than full models.

We are specifically interested in problems with non-trivial sparse relationships amongst the data. In particular, problems where the dependencies exhibit some sparse patterns that can be exploited in the modelling but for which the prior understanding is not sufficient to formulate explicit constraints to be hard-wired into the model. We build on the ideas of learning with structured sparsity to factor such patterns into the models.

Furthermore, as the relationships may be too complex to be satisfactorily captured by simple linear functions we allow the methods to operate over a broader space of nonlinear functions. For this we rely on the theory of regularised learning in the reproducing kernel Hilbert spaces (RKHSs) and extend it in the direction of sparse learning in nonlinear non-additive models. Throughout the thesis we propose multiple new methods for sparse learning over reduced set of input variables. We initially concentrate on the problem of multivariate time series forecasting and develop methods that learn forecasting models together with discovering the Granger causality dependencies amongst the series.

In the second half of the manuscript we focus on the more general problem of learning sparse nonlinear regression functions. Making parallels to linear modelling, we formulate new regularisers based on partial derivatives of the function to promote structured sparsity in the nonlinear model. We show how these can be incorporated into the kernel regression problem and reformulated into a problem solvable in practice by an iterative algorithm derived from the alternating direction method of multipliers (ADMM).

Finally, we address the scalability issues of sparse learning with kernel methods. We use the random Fourier features to approximate the kernel function and shift the sparsity search from the original function space into the space of the random features. We thus significantly reduce the dimensionality of the search space and therefore the computational complexity even when working over large datasets with thousands of data instances.