Sujet de Thèse
• Titre : Towards new algorithms for sparse approximation in continuous dictio- naries
• Unité de recherche :IRMAR, UMR-6625
• Thème :Analyse
• Mots clefs : sparse representations, parametric dictionaries, combinatorial opti- mization, screening methods.
• Les noms, prénoms, courriel, établissement des directeurs ou directrices de thèse
1. Herzet, Cédric, [email protected], INRIA
2. Elvira, Clément, [email protected], CentraleSupelec
Objectif de la thèse
This thesis focuses on the paradigm of sparse representations in “continuous” dictionar- ies. This topic of research is one of the latest developments in the theory of compressive sensing and promises to provide new theoretical and algorithmic answers to the field of signal processing and statistics.
While mainstream research in compressive sensing considers dictionaries obtained by discretizing continuously-valued problems on fine grids, this approach completely fails in many setups encountered in physics, biology, chemistry, etc. Indeed, on the one hand, the grid resolution needed to solve problems to the desired accuracy is often very fine; this leads to discretized problems intractable to handle. On the other hand, discretization introduces some model distorsion which impacts the performance of the reconstruction algorithms (even when no noise corrupts the data, see [1]).
Continuous dictionaries circumvent these issues by considering atoms (i.e., the el- ementary signals used to “sparsely” describe the data) continuously indexed by some low-dimensional parameters [2]. There is therefore no discretization here but the de- composition dictionary contains an infinite uncountable number of elements.
So far sparse representations in continuous dictionaries have been mainly explored through the use of convex relaxation methods: finding a sparse representation of some input signal is expressed as some optimization problem over the space of Radon mea- sures, seee.g., [3, 4].1 Although this approach enjoys algorithmic and analytical ad- vantages, its “relaxed” nature also induces limitations in terms of signal reconstruction and identification, see [2].
The goal of this thesis is to leverage recent developments in operational research to address the (nonconvex) sparse approximation problem in continuous dictionaries. A first avenue of research will be to extend the notion ofsafe screening ruleintroduced in [5] for the LASSO problem to the continuous setting. More specifically, the PhD stu- dent will be asked to derive tractable methods able to detect groups of atoms that will notbe used in the optimal sparse decomposition. Once such a set of irrelevant atoms is detected, it can be discarded from the feasible set, thus enabling potentially significant speedup of optimization procedures. Yet, exploiting this information in presence of in- finite uncountable dictionaries remains an open question. In this task, the PhD student
1This problem is known as “Beurling LASSO” and is an extension of the well-known LASSO problem to the context of continuous dictionaries.
1
will explore the recent concepts of “nonconvex screening” [6] and “joint screening” [7].
The PhD student will then study how screening and finite-dimensional combinatorial optimization methods can be intertwined to solve the infinite-dimensional sparse ap- proximation.
References
[1] Vincent Duval and Gabriel Peyré, “Exact support recovery for sparse spikes decon- volution,” Foundations of Computational Mathematics, vol. 15, no. 5, pp. 1315–
1355, 2015.
[2] Emmanuel J. Candès and Carlos Fernandez-Granda, “Towards a mathematical the- ory of super-resolution,”Comm. Pure Appl. Math, vol. 67, no. 6, pp. 906–956, June 2014.
[3] Kristian Bredies and Hanna Katriina Pikkarainen, “Inverse problems in spaces of measures,” ESAIM: Control, Optimisation and Calculus of Variations, vol. 19, no.
1, pp. 190–218, 2013.
[4] Yohann de Castro and Fabrice Gamboa, “Exact reconstruction using Beurling min- imal extrapolation,” Journal of Mathematical Analysis and Applications, vol. 395, no. 1, pp. 336 – 354, 2012.
[5] Laurent El Ghaoui, Vivian Viallon, and Tarek Rabbani, “Safe feature elimination in sparse supervised learning,” Tech. Rep. UC/EECS-2010-126, EECS Dept., Uni- versity of California at Berkeley, Sept. 2010.
[6] Alper Atamturk and Andres Gomez, “Safe screening rules for l0-regression from perspective relaxations,” inProceedings of the 37th International Conference on Machine Learning, Virtual, 13–18 Jul 2020, vol. 119 of Proceedings of Machine Learning Research, pp. 421–430.
[7] Cédric Herzet, Clément Dorffer, and Angélique Drémeau, “Gather and conquer:
Region-based strategies to accelerate safe screening tests,” IEEE Transactions on Signal Processing, vol. 67, no. 12, pp. 3300–3315, 2019.
2
