Gradient descent
Application of Metamodel-Assisted Multiple-Gradient Descent Algorithm (MGDA) to Air-Cooling Duct Shape Optimization
... 1 INTRODUCTION MGDA stands for Multiple-Gradient Descent Algorithm [1]. It is a generalization of the classical steepest-descent method [4] that applies to cases in which an arbitrary number of cri- ...
11
Stochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descent
... N gradient evaluations per step which is prohibitively large in applications in deep learning that involve networks with many parameters (tenths of thousands up to ...Stochastic Gradient Descent ...
16
On the almost sure convergence of stochastic gradient descent in non-convex problems
... Key words and phrases. Non-convex optimization; stochastic gradient descent; stochastic approximation. This research was partially supported by the COST Action CA16228 “European Network for Game Theory” ...
33
A cooperative algorithm for multi-objective optimization: multiple-gradient descent algorithm (MGDA)
... From a given starting point, MGDA converges quickly (6 steps in this example) and provides an accurately-defined design-point on the Pareto set. After applying the method from a set of 60 initial design-points ...
9
A Gradient-Descent Method for Curve Fitting on Riemannian Manifolds
... piecewise arcs. The panels (a) and (b) show examples of optimal γ for N = 2 (three data points) and N = 3 (four data points) with λ values being 100 and 1, respectively. For λ = 100, the resulting optimal curve looks ...
23
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
... stochastic gradient methods have proven to be very effective thanks to their low iteration ...Stochastic Gradient descent (SGD, ( Robbins and Monro , 1951 )) and Stochastic Heavy Ball (SHB, ( Polyak ...
34
Convergence of constant step stochastic gradient descent for non-smooth non-convex functions
... Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non convex, non smooth, locally Lipschitz random ...the gradient may not exist, it is ...
32
Phase Code Optimization for Coherent MIMO Radar Via a Gradient Descent
... † SONDRA – CentraleSupélec, 91192 Gif-Sur-Yvette cedex, France ‡ ONERA, The French Aerospace Lab, 91123 Palaiseau cedex, France Abstract—In this paper, a gradient descent method is used to build radar ...
7
Phase Code Optimization for Coherent MIMO Radar Via a Gradient Descent
... † SONDRA – CentraleSupélec, 91192 Gif-Sur-Yvette cedex, France ‡ ONERA, The French Aerospace Lab, 91123 Palaiseau cedex, France Abstract—In this paper, a gradient descent method is used to build radar ...
7
Noise-aided gradient descent bit-flipping decoders approaching maximum likelihood decoding
... In this paper, we will study two recently introduced LDPC decoders derived from noisy versions of the gradient descent bit-flipping decoder (GDBF). Although the GDBF is known to be a simple decoder with ...
6
Multiple-Gradient Descent Algorithm (MGDA) for Pareto-Front Identification
... Multiple-Gradient Descent Algorithm (MGDA) for Pareto-Front Identification Jean-Antoine D´esid´eri Abstract This article compounds and extends several publications in which a Multiple-Gradient ...
18
Quasi-Riemannian Multiple Gradient Descent Algorithm for constrained multiobjective differential optimization
... the descent direction is determined by the Multiple-Gradient Descent Algorithm (MGDA) applied to the cost- function gradients projected onto the subspace locally tangent to all constraint ...
45
Stochastic Gradient Descent on a Portfolio Management Training Criterion Using the IPA Gradient Estimator
... Since we have Lipschitz continuity a.s., the dominated convergence theorem applies and we may, therefore, perform a stochastic gradient descent and use backpropagation to evaluate the dθ d U θ IPA ...
19
Multiple-Gradient Descent Algorithm (MGDA)
... the gradient-vectors is nonzero, and −ω is a descent direction for all criteria ...Multiple-Gradient Descent Algorithm (MGDA) which generalizes the classical steepest-descent algorithm ...
20
A zealous parallel gradient descent algorithm
... asynchronous gradient descent and propose a zealous variant that minimizes the idle time of processors to achieve a substantial speedup. We then experimentally study this algorithm in the context of ...
1
Parametric optimization of pulsating jets in unsteady flow by Multiple-Gradient Descent Algorithm (MGDA)
... Multiple-Gradient Descent Algorithm (MGDA) was originally introduced in [1] and [2] to solve general multi-objective optimization problems involving differ- entiable ...
18
MUTIPLE-GRADIENT DESCENT ALGORITHM FOR MULTIOBJECTIVE OPTIMIZATION
... Multiple-Gradient Descent Algorithm (MGDA ), originally introduced in [3], and again formal- ized in [5], is based on a very general principle permitting to define at each iteration, a descent ...
21
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise
... Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization ...the gradient noise in several deep learning settings ...
12
Fast Non Rigid Matching by Gradient Descent: Study and Improvements of the "Demons" Algorithm
... Fast Non Rigid Matching by Gradient Descent: Study and Improvements of the ”Demons” Algorithm Pascal Cachier, Xavier Pennec, Nicholas Ayache.. To cite this version: Pascal Cachier, Xavie[r] ...
29
Anderson acceleration of coordinate descent
... to gradient and proximal gradient ...coordinate descent achieves perfor- mance significantly superior to full-gradient ...coordinate descent in practice is not easy: inertially ...
16