Identification · Local Linear Convergence

Top PDF Identification · Local Linear Convergence:

Model identification and local linear convergence of coordinate descent

Model identification and local linear convergence of coordinate descent

Equal contribution October 16, 2020 Abstract For composite nonsmooth optimization problems, Forward-Backward algorithm achieves model identification (e.g., support identification for the Lasso) after a finite number of iterations, provided the objective func- tion is regular enough. Results concerning coordinate descent are scarcer and model identification has only been shown for specific estimators, the support-vector machine for instance. In this work, we show that cyclic coordinate descent achieves model identification in finite time for a wide class of functions. In addition, we prove explicit local linear convergence rates for coordinate descent. Extensive experiments on various estimators and on real datasets demonstrate that these rates match well empirical results.
En savoir plus

27 En savoir plus

Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems

Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems

(PS) penalties and general C 2 loss functions with Lipschitz gra- dient. Under nonlinear complementarity requirements analogous to the non-degeneracy assumption “(x ∗ ) > 0” of Theorem 1(b), and rank constraints analogous to the requirement that the Jacobian matrix T 0 (x ∗ ) have spectral radius less than 1 (in Theorem 1(c2)), the authors of [22, 23] prove finite-time activity identification and local Q-linear convergence at a rate given in terms of Friedrichs angles, via direct application of [20, Theorem 3.10]. The authors show that their arguments are valid for a broad variety of problems, for example the anisotropic TV penalty. Still in the framework of partly-smooth penalties, [25] showed local Q-linear convergence of the Douglas-Rachford algorithm on the Basis Pursuit problem. Comparison with [22, 23]. The works which are most compara- ble to ours are [22] and [23], already presented above. Let us point out some similarities and differences between these papers and ours. First, though our constructions are entirely different from the tech- niques developed in [22, 23], one notes that both approaches are ultimately rooted in the same idea, namely the work of B. Holmes [16] on the smoothness of the euclidean projection onto convex sets, and other related functionals (Minkowski gauges, etc.). Indeed, The- orem 1 builds directly upon [16], whilst, [23] and [22] are linked to [16] via [26], which builds on [27], and the latter builds on [16].
En savoir plus

6 En savoir plus

Identification en temps fini et convergence linéaire locale de l'algorithme proximal implicite-explicite inertiel

Identification en temps fini et convergence linéaire locale de l'algorithme proximal implicite-explicite inertiel

Abstract – We consider the class of inertial Forward–Backward (iFB) proximal splitting algorithms, to minimize the sum of two proper lower semi-continuous convex functions, one of which having a Lipschitz continuous gradient and the other being partly smooth relative to an active manifold M. Special cases of this class include the FB and, for an appropriate choice of the inertial parameter, FISTA-like schemes. We propose a unified analysis, under which we show that iFB-type splitting, (i) correctly identifies the active manifold M in a finite number of iterations, and then (ii) enters a local (linear) convergence regime, which is characterised precisely. This gives a grounded justification to the typical behaviour that has been observed numerically for many problems encompassed in our framework, including the Lasso, the group Lasso, total variation minimization and the nuclear norm regularization to name a few. These results may have numerous applications including in signal/image processing processing and machine learning.
En savoir plus

4 En savoir plus

Local Linear Convergence of Inertial Forward-Backward Splitting for Low Complexity Regularization

Local Linear Convergence of Inertial Forward-Backward Splitting for Low Complexity Regularization

predictions. The beginning of the dashed lines are the points when x k identifies the manifold M x ⋆ . As one can observe, FISTA has the fastest manifold identification, however, locally it is the slowest for all tested examples. Indeed, when the manifold is affine, it can be shown from Theorem III.1 that ρ k ∈]η k , √η k ] for a k > η k , i.e. FISTA is locally slower than FB.

2 En savoir plus

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

13. Liang, J., Fadili, M.J., Peyré, G., Luke, R.: Activity identification and local linear convergence of Douglas–Rachford/ADMM under partial smoothness. In: Scale Space and Variational Methods in Computer Vision, pp. 642–653. Springer (2015) 14. Borwein, J.M., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: H.H. Bauschke, R.S. Burachik, P.L. Combettes, V. Elser, D.R. Luke, H. Wolkowicz (eds.) Fixed-Point Al- gorithms for Inverse Problems in Science and Engineering, Springer Optimization and Its Applications, vol. 49, pp. 93–109. Springer New York (2011)
En savoir plus

34 En savoir plus

A Unifying Local Convergence Result for Newton's Method in Riemannian Manifolds

A Unifying Local Convergence Result for Newton's Method in Riemannian Manifolds

Abstract: We consider the problem of finding a singularity of a vector field X on a complete Riemannian manifold. In this regard we prove a unified result for local convergence of Newton’s method. Inspired by previous work of Zabrejko and Nguen on Kantorovich’s majorant method, our approach relies on the introduction of an abstract one-dimensional Newton’s method obtained using an adequate Lipschitz-type radial function of the covariant derivative of X. The main theorem gives in particular a synthetic view of several famous results, namely the Kantorovich, Smale and Nesterov-Nemirovskii theorems. Concerning real-analytic vector fields an application of the central result leads to improvements of some recent developments in this area.
En savoir plus

31 En savoir plus

Optimal linear drift for the speed of convergence of an hypoelliptic diffusion

Optimal linear drift for the speed of convergence of an hypoelliptic diffusion

For both dynamics (2) and (3) it is dicult for a general potential V to obtain sharp theoretical bounds on the rates of convergence (see [13] for consideration on this matter in the metastable case, namely the regime " ! 0 with the potential V " = 1 " V where V has several local minima). A particular simple situations is the case where V is quadratic or, in other words,  is a Gaussian measure. Of course MCMC algorithms are not really relevant in practice regarding sampling according to Gaussian measures, but then the exact rates of convergence for (2) and (3) are trackable (see [2, 12] and below).
En savoir plus

15 En savoir plus

Local church and Church universal: towards a convergence between East and West

Local church and Church universal: towards a convergence between East and West

Formed as a historian and canonist, the young lay theologian began taking interest in dogmatics: an interest awakened in part by his study of Ibas of Edessa and in part by his contact with Father Bulgakov. 38 It seems that his studies in philosophy, history, canon law, New Testament, dogmatics, etc. transformed him from a canonist into an ecclesiologist and his lessons in canon law—that dry subject that he was asked to teach—into lessons of ecclesiology. This transformation is visible in the two articles he wrote between 1931 and 1932: “The Two conceptions of the universal Church,” 39 “The Canons and Canonical Conscience.” 40 The publication of the first of these articles marked a landmark in the theological career of Afanasiev. In this article, originally prepared as an exposition for Father Bulgakov’s ‘Seminar,’ 41 we find the first sketches of his Eucharistic Ecclesiology. Distinguishing between the two conceptions of the universal Church in vigour—that of Rome (a juridical universality around the successor of Peter) and that of Constantinople (ecumenical universality)—the author states that in both these concepts the catholicity of the Church is understood in quantitative terms, whereas in reality it is a qualitative reality which has its foundation in the Eucharist. As he saw it, the unity of the Church is the unity of the local Churches, which are united to one another through their communion to the one and unique Table of the Lord. Contrary to the expectations of the author, the article was accorded only a cold reception by his mentor, S. Bulgakov, because the latter did not find any relationship between the Eucharistic Ecclesiology and the doctrine of Sobornost’. Rather discouraged, Afanasiev decided to return to his earlier preoccupations, viz. canonical and historical questions. Thus, during the years preceding the World War II, he launched an ambitious project of writing a work on Ecclesial Councils and Their Origin. But the outbreak of the war interrupted the work.
En savoir plus

585 En savoir plus

Subspace Identification for Linear Periodically Time-varying Systems

Subspace Identification for Linear Periodically Time-varying Systems

able, whereas, the second approach suggests to find a set of output subsequences that have time-invariant behaviors. This makes it possible to derive an identification algorithm for these subsequences, which is close to the classical time- invariant algorithm. Among the few attempts on the sub- ject, one can cite Liu [1997] where a discrete time domain state space model, transition matrices and pseudo-modal parameters are used to describe and identify recursively periodic systems, and Verhaegen and Yu [1995] in which subspace model identification algorithms that allow the identification of an LPTV state space model from a set of input-output measurements are presented.
En savoir plus

8 En savoir plus

Combflex: a linear combinatorial auction for local energy markets

Combflex: a linear combinatorial auction for local energy markets

i t 2 , ∀ t 2 > t 1. Assumption 1 is satisfied by most residential tariffs and feed-in-tariffs (FIT, the price at which prosumers can inject their surplus energy into the grid), and it guarantees that storing energy in the battery for selling it later is never optimal. Assumption 2. Since players have inflexible usage of their appliances, their utility v i is set to be 0 when consuming their desired load profile , and it is set to −∞ if they do not. Fur- thermore, users have quasi-linear utilities: u i (z i ) = v i (z i )−p. Because the load will always be satisfied, their utility can be summarized by the negative amount of money that they need to pay for electricity.
En savoir plus

8 En savoir plus

Local convergence of large random triangulations coupled with an Ising model

Local convergence of large random triangulations coupled with an Ising model

3-LQG. Such a convergence has been established in the case of “pure quantum gravity”, corresponding to uniform planar maps and γ = p 8/3, in the impressive series of papers by Miller and Sheffield [57, 58, 59]. Obtaining such a result for a model of decorated maps outside the pure-gravity class seems out of reach for the moment. However – building on the so-called mating-of-trees approach initiated by Sheffield [61] and which has allowed to obtain various local convergence results for models of decorated maps (see e.g.[27, 12, 44, 46, 45]) – Gwynne, Holden and Sun [42] managed to prove that for some models of decorated maps, including the spanning-tree decorated maps, bipolar oriented maps and Schnyder wood decorated maps, the volume growth of balls in their local limit is given by the “fractal dimension” d γ , for the
En savoir plus

43 En savoir plus

Identification of switched linear systems via sparse optimization

Identification of switched linear systems via sparse optimization

a fraction of the nonzero entries over the total length of the vector φ(θ 3 ). 4.2 Identification of the PVs The second objective is to test the statistical robustness of the identification algorithm. For this purpose, we use 100 different independent realizations of the input, the discrete state and the output noise (SNR=30 dB) to generate 100 data sequences of length N = 600 each. The identification algorithm (Algorithm 2 indeed) is then run on each of these different 100 data sequences. The user- specified parameters of Algorithm 1 and Algorithm 2 are set to ε = 0.1, Tol = 0.001 and Thres = 0.05. At each run, the first 300 points are used to identify a model and the whole sequence of length 600 is used to validate the estimated model, i.e., to verify its ability to reconstruct the system output from the true input and an estimated discrete state. This is evaluated with the criterion [20]
En savoir plus

15 En savoir plus

CONSTRUCTION AND CONVERGENCE STUDY OF SCHEMES PRESERVING THE ELLIPTIC LOCAL MAXIMUM PRINCIPLE

CONSTRUCTION AND CONVERGENCE STUDY OF SCHEMES PRESERVING THE ELLIPTIC LOCAL MAXIMUM PRINCIPLE

Remark 4.1. Obviously, the scheme being nonlinear (which is nearly a require- ment for monotone methods; see the introduction), the computation of its solution is more costly than for linear methods. We will nevertheless see that, in many tests, the number of iterations in the algorithm stays relatively low; it should also be noticed that the linear system solved at each iteration comes from an M -matrix that is roughly as well conditioned as the matrices of usual linear schemes and is also well adapted to multigrid methods. Finally, we notice that realistic models (see section 4.2.3) are non- linear and that, in practice, the “additional” nonlinearity introduced by our nonlinear method increases very little the cost of solving the scheme.
En savoir plus

33 En savoir plus

Identification of space deformation using linear and superficial quadratic variations

Identification of space deformation using linear and superficial quadratic variations

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignemen[r]

13 En savoir plus

Identification-robust inference for endogeneity parameters in linear structural models

Identification-robust inference for endogeneity parameters in linear structural models

(4) confidence sets for a = θ − β and any linear transformation w ′ a may then be derived by projec- tion; these confidence sets have level 1 − α ; (5) confidence sets for σ Vu and w ′ σ Vu can finally be built using the relationship σ Vu = Σ V a . For inference on a, we develop a finite-sample approach which remains valid irrespective of as- sumptions on the distribution of V. In addition, we observe that the test statistics used for inference on β [the AR-type statistic] and θ enjoy invariance properties which allow the application of Monte Carlo test methods: as long as the distribution of the errors u is specified up to an unknown scale parameter, exact tests can be performed on β and θ through a small number of Monte Carlo simula- tions [see Dufour (2006)]. For inference on both regression and covariance endogeneity parameters
En savoir plus

34 En savoir plus

On-line structured subspace identification with application to switched linear system

On-line structured subspace identification with application to switched linear system

Most of the existing contributions on the subject deal with the class of Piecewise ARX models. These models are defined on a polyhedral partition of the regression space, each submodel being associated to one polyhedron. Hence, the main challenge in the identifica- tion of this class of systems is to determine the right partition of the regressors. Once this task is complete, the estimation of the different submodels follows by means of standard linear regression techniques [4]. To find the regions, it is proposed in [1] to group the regressors associated with each mode by performing the K-means clustering algorithm in a special space and then, to compute the parameters of each submodel. The solution in addi- tion to requiring a knowledge of the system order is suboptimal. In a similar framework, [5] uses a statistical clustering approach instead and provides a method to derive the number of submodels from batch data, the orders being assumed to be available a priori. Another category of methods alternates between assigning the data to submodels and estimating simultaneously their parameters by using a weights learning technique [6], solving a Mini- mum Partition into Feasible Subsystems (MIN PFS) problem [7], or resorting to bayesian learning [8]. In [9], the hybrid system identification problem has been transformed into a linear or quadratic mixed integer programming problem for which there exist efficient tools for solving it in an optimal way. Conversely, this algorithm suffers from a high computa- tional complexity. Another optimal, but deterministic algorithm is the algebraic geometric approach developed in [3]. Under the assumption that the data are perfect (in the sense that they are not corrupted by noise), the authors recast the problem into one of computing and deriving a homogeneous polynomial from which the submodels are deduced without any iteration [10]. For a comprehensive review of hybrid systems identification methods, we refer the interested reader to the survey paper [11].
En savoir plus

36 En savoir plus

Identification of porcine diaphragm non-linear behavior via bulge tests

Identification of porcine diaphragm non-linear behavior via bulge tests

Figure 2: Example of inverse identification with 4 constitutive models Discussion The central tendon membrane of porcine clearly showed a non-linear behavior with a characteristic “toe-region”, similar to those obtained for many other soft tissue behaviors.

3 En savoir plus

LOCAL WELL-POSEDNESS FOR THE QUASI-LINEAR HAMILTONIAN SCHRÖDINGER EQUATION ON TORI

LOCAL WELL-POSEDNESS FOR THE QUASI-LINEAR HAMILTONIAN SCHRÖDINGER EQUATION ON TORI

The Hypothesis 1.1 is needed in order to cover the case of large initial conditions, this is compatible with the global ellipticity condition we assumed in [8] and with the ones given in [13, 17]. As already said, this hypothesis is not necessary in the case of small data. We discuss briefly the strategy of our proof. We begin by performing a para-linearization of the equation `a la Bony [3] with respect to the variables (u, u). Then, in the same spirit of [8], we construct the solutions of our problem by means of a quasi-linear iterative scheme `a la Kato [11]. More precisely, starting from the para-linearized system, we build a sequence of linear problems which converges to a solution of the para-linearized system and hence to a solution of the original equation (1.1). At each step of the iteration one needs to solve a linear para-differential system, in the variable (u, u), with non constant coefficients (see for instance (4.93)). We prove the existence of the solutions of such a problem by providing a priori energy estimates (see Theorem 4.1). In order to do this, we diagonalize the system in order to decouple the dependence on (u, u) T up to order zero. This is done by applying changes of coordinates generated by para-differential operators. Once achieved such a diagonalization we are able to prove energy estimates in an energy-norm, which is equivalent to the Sobolev norm.
En savoir plus

31 En savoir plus

Activités économiques et intérêt local : mesure des flux et identification des localisations optimales

Activités économiques et intérêt local : mesure des flux et identification des localisations optimales

légales de calcul et des statistiques. Ce n’est que lorsque cette différence est mesurée jusqu’au 31 décembre que l’impact fiscal pourrait être mesuré. Dans le cas d’une création d’entreprise et d’emplois, par exemple via la création d’une zone d’activités économiques, l’évaluation de l’impact économique et fiscal attendu sera plus complexe et peut être insuffisamment scientifiquement maîtrisable : à l’estimation de la différence (ici, positive et sur une durée annuelle) de salaire, il faudra ajouter un facteur peu évident à cerner : celui de la part des postes qui pourraient être occupés par des personnes habitant le territoire communal ou local. Les facteurs qui influencent chez l’employeur l’engagement de personnes domiciliées sur la commune dépendent de facteurs plus ou moins concrets (les personnes en recherche d’emploi ont-elles les capacités requises ?) mais aussi souvent de facteurs moins perceptibles ou plus subjectifs (la stratégie RH de l’entreprise, qui peut délibérément et stratégiquement choisir de recruter ailleurs, la capacité de négociation de l’élu local…).
En savoir plus

34 En savoir plus

Identification of Time-Non Local Models under Diffusive Representation

Identification of Time-Non Local Models under Diffusive Representation

an operator means identifying one of its characteristic quantities. As the operator to be identified is linear, a convenient and rather general approach consists in working in the frequency domain where any causal operator can be well-defined by its symbol H(iω), that is the Fourier transform of its impulse response. Then, the problem of identifying H(iω) can be classically solved from physical measurements by means of Fourier techniques. Note how- ever that purely frequency identification presents some well-known shortcomings. In particular, the so-identified symbol H(iω) is in general ill adapted to the construction of efficient time-realizations. This is partly due to exces- sive numerical cost of quadrature approximations resulting from the intrinsic convolution nature of the associated operator, sometimes with long memory (Rumeau et al. [2006]) or even delay-like behaviors (Montseny [2007]). Another shortcoming is that frequency methods are incom- patible with real-time identification (and so with pursuit when the symbol has the ability to evolve slowly). But above all, the number of unknown parameters is excessive, what makes the problem excessively sensitive to measure- ment noises.
En savoir plus

9 En savoir plus

Show all 8056 documents...