split-and-augmented Gibbs sampler

Top PDF split-and-augmented Gibbs sampler:

Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler

Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler

Logistic regression has been extensively used to perform classifica- tion in machine learning and signal/image processing. Bayesian for- mulations of this model with sparsity-inducing priors are particularly relevant when one is interested in drawing credibility intervals with few active coefficients. Along these lines, the derivation of efficient simulation-based methods is still an active research area because of the analytically challenging form of the binomial likelihood. This paper tackles the sparse Bayesian binary logistic regression problem by relying on the recent split-and-augmented Gibbs sampler (SPA). Contrary to usual data augmentation strategies, this Markov chain Monte Carlo (MCMC) algorithm scales in high dimension and di- vides the initial sampling problem into simpler ones. These sam- pling steps are then addressed with efficient state-of-the-art methods, namely proximal MCMC algorithms that can benefit from the recent closed-form expression of the proximal operator of the logistic cost function. SPA appears to be faster than efficient proximal MCMC al- gorithms and presents a reasonable computational cost compared to optimization-based methods with the advantage of producing cred- ibility intervals. Experiments on handwritten digits classification problems illustrate the performances of the proposed approach.
En savoir plus

7 En savoir plus

Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler

Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler

Logistic regression has been extensively used to perform classifica- tion in machine learning and signal/image processing. Bayesian for- mulations of this model with sparsity-inducing priors are particularly relevant when one is interested in drawing credibility intervals with few active coefficients. Along these lines, the derivation of efficient simulation-based methods is still an active research area because of the analytically challenging form of the binomial likelihood. This paper tackles the sparse Bayesian binary logistic regression problem by relying on the recent split-and-augmented Gibbs sampler (SPA). Contrary to usual data augmentation strategies, this Markov chain Monte Carlo (MCMC) algorithm scales in high dimension and di- vides the initial sampling problem into simpler ones. These sam- pling steps are then addressed with efficient state-of-the-art methods, namely proximal MCMC algorithms that can benefit from the recent closed-form expression of the proximal operator of the logistic cost function. SPA appears to be faster than efficient proximal MCMC al- gorithms and presents a reasonable computational cost compared to optimization-based methods with the advantage of producing cred- ibility intervals. Experiments on handwritten digits classification problems illustrate the performances of the proposed approach.
En savoir plus

8 En savoir plus

Split-and-augmented Gibbs sampler - Application to large scale inverse problems

Split-and-augmented Gibbs sampler - Application to large scale inverse problems

amount and variety of available data, solving such inference problems in high dimension be- comes challenging and generally relies on sophisticated computational inference methods. Those methods are mainly based on stochastic simulation and variational optimization which are two powerful tools to perform inference in complex models [2]. An important class of stochastic simulation techniques is the family of the Markov chain Monte Carlo (MCMC) methods [3]. Within a Bayesian inference framework, MCMC algorithms have the great advantage of providing a comprehensive description of the posterior distribution of the parameter x to be inferred. Contrary to optimization techniques which generally provide a point estimate, this description permits the subsequent derivation of credibility intervals on the parameter x. Nonetheless, note that optimization algorithms can also bring confidence information when the log-likelihood is supposed differentiable by relying on the theory of large samples [4]. These confidence measures are particulary important for inference problems where very few observations are available (e.g. in biology [5], physics [6] or astrophysics [7]) or when one is interested in extreme events (e.g. in hydrology [8] or cosmology [9]). For instance, MCMC methods have been recently used to conduct Bayesian inference on gravitational waves [10]. However, contrary to optimization techniques, MCMC methods may suffer from their high computational cost which can be prohibitive for high-dimensional problems. To overcome this limitation, a few attempts have been made to derive optimization-driven Monte Carlo methods. The Hamiltonian Monte Carlo method [11], also referred to as hybrid Monte Carlo, is an archetypal example of the successful use of variational analysis concepts (i.e., gradients) to facilitate the exploration of the target distribution. More recently, Pereyra [12] proposed an innovative combination of convex optimization and MCMC algorithms. Capitalizing on the advantages of proximal splitting recently popularized to solve large-scale inference problems [13]–[18], the proximal Monte Carlo method allows high-dimensional log-concave distributions to be sampled. For instance, this algorithm has been successfully used to conduct antisparse coding [19] and has been significantly improved in [20]. Concurrently, variable splitting methods, developed at least 70 years ago [21], have been recently and extensively used to solve large-scale inference problems of the form
En savoir plus

35 En savoir plus

Split-and-augmented Gibbs sampler - Application to large-scale inference problems

Split-and-augmented Gibbs sampler - Application to large-scale inference problems

It is worth noting that this variable splitting-based approach can be related to previous works [32], [33], revisited and extended in [34], which also introduced auxiliary variables to split the initial objective function. However, the aforementioned works considered an exact data augmentation scheme which is not the case here, see Theorem 1 below. In addition, this scheme was specifically designed for Bayesian models relying on a Gaussian likelihood function, which is much more restrictive than the target distribution (3) addressed here. Finally, the data aug- mentation scheme considered in [32], [33] may practically rise some computational difficulty since it requires closed-form expressions of the augmented prior, which could not be available in general. Nonetheless, note that both the latter and the proposed approaches can be interpreted as divide-to-conquer approaches ending up with simpler full conditional distributions.
En savoir plus

35 En savoir plus

Split rank of triange and quadrilateral inequalities

Split rank of triange and quadrilateral inequalities

Most inequalities used in commercial softwares are split cuts Question : what is the split rank of the 2 row-inequalities ? In how many rounds of split cuts only can we generate the inequalities ? The Cook-Kannan-Schrijver has infinite rank and we prove that the other triangles have finite rank.

87 En savoir plus

Probabilistic coloring of bipartite and split graphs

Probabilistic coloring of bipartite and split graphs

Dealing with   in bipartite graphs, it is shown in [17] that it is NP-hard even if the input has only four distinct vertex-probabilities with one of them being equal to 0. Moreover, a poly- nomial algorithm was devised, achieving approximation ratio bounded above by 2.773. The NP-hardness result of [17] left, however, several open questions. For instance, “what is the complexity of   when we further restrict inputs, say in paths, or trees, or cycles, or stars, . . . ?”, etc. In this paper, we prove that, under non-identical vertex-probabilities,   is polynomial for stars and for trees with bounded degree and a fixed number of distinct vertex-probabilities and we deduce as a corollary that it is polynomial also for paths with a fixed number of distinct vertex-probabilities. Then, we show that, assuming identical vertex-probabilities, the problem is polynomial for paths, for even and odd cycles and for trees all leaves of which are either at even or at odd levels. We finally focus ourselves on split graphs and show that, in such graphs,   is NP-hard, even assuming identical vertex probabilities.
En savoir plus

21 En savoir plus

Mobile Collaborative Augmented Reality: The Augmented Stroll

Mobile Collaborative Augmented Reality: The Augmented Stroll

Information from the real environment is transferred to the virtual world thanks to the camera carried by the user. The camera is positioned so that it corresponds to what the user is seeing, through the HMD. The real environment captured by the camera can be displayed in the gateway window on the pen computer screen as a background. Information from the virtual world is transferred to the real environment, via the gateway window, thanks to the HMD. For example the archaeologist can drag a drawing or a picture stored in the database to the gateway window. The picture will automatically be displayed on the HMD on top of the real environment. Moving the picture using the stylus on the screen will move the picture on top of the real environment. This is for example used by archaeologists in order to compare objects, one from the database and one just discovered in the real environment. In addition the user can move a cursor in the gateway window that will be displayed on top of the real environment. The ultimate goal is that the user can interact with the real environment as s/he does with virtual objects. Based on this concept of gateway between the real and the virtual, we implemented the clickable reality and the augmented stroll. We first describe the clickable reality and the augmented stroll and then focus on their software design.
En savoir plus

20 En savoir plus

Split rank of triangle and quadrilateral inequalities

Split rank of triangle and quadrilateral inequalities

+ ) | f + Rs = x}, (1) where f ∈ Q 2 \ Z 2 and R = [r 1 , r 2 , ..., r k ] ∈ Q 2×k . These inequalities are either split cuts or intersection cuts (the so called triangle and quadrilateral inequalities). The motivation for studying P (R, f ) is the following: Given two rows of a simplex tableau corresponding to integer basic variables that are at fractional values, P (R, f ) is obtained by relaxing the non-basic integer variables to be continuous variables and by relaxing the basic integer variables to be free integer variables. As P (R, f ) can be obtained as a relaxation of any mixed integer program, valid inequalities for the convex hull of P (R, f ) can be used as a source of cutting planes for general mixed integer programs. Empirical experiments with some classes of related cutting planes by Espinoza [21] present evidence that these new inequalities may be useful computationally. Various extensions to the basic relaxation P (R, f ) have also been recently studied where the inequalities are related to triangles and quadrilaterals; see for example Dey and Wolsey [18], Andersen et al. [1], Dey and Wolsey [20], Basu et al. [9], Conforti et. al [11] and Fukasawa and G¨ unl¨ uk [22].
En savoir plus

50 En savoir plus

Treating the Gibbs phenomenon in barycentric rational interpolation and approximation via the S-Gibbs algorithm

Treating the Gibbs phenomenon in barycentric rational interpolation and approximation via the S-Gibbs algorithm

I, which we denote by X rand . Looking at Table 1 , we observe that using the AAA algorithm with starting set S( X rand ) (indicated in the Table as AAA S ), that is, constructing the approximants via the fake nodes approach, does not suffer from the effects of the Gibbs phenomenon. For both approximants we fix the maximum degree to 20 and to 40 (by default 100 in the algorithm).

7 En savoir plus

New Augmented Reality Taxonomy: Technologies and Features of Augmented Environment.

New Augmented Reality Taxonomy: Technologies and Features of Augmented Environment.

whose final aim is purely aesthetic. Designers therefore have greater relative freedom and are not limited to the types of environments which they can use: they can go beyond the possibilities that this environment really offers and the means used may be very varied. Concretely, it is possible to use all types of environments in the augmented reality perception functionality . Without systematically and exhaustively listing all the types of artificial mixed envi- ronments, it is possible to give a few examples. The principle for all possible sub-functionalities consists of diverting the primary function of AR, ie. aug- mented perception of reality. Creating an artificial AR environment can for example be envisaged thanks to a semantic gap produced by the incoherence of the overall meaning of the mixed environment. Diverting sub-functionality n ◦ 1 (documented reality and documented virtuality) of the first functional- ity (augmented perception of reality) involves associating, for example, with a real environment a virtual “document” providing information out of step with this environment with the aim of provoking in users a modification of the meaning of reality as it is perceived by users due to the difference between the real image and virtual information. We could for example as- sociate with a real environment a virtual “document” erroneously describing this environment. Synthesising these mutually incompatible meanings would provoke in users the representation of an impossible mixed artificial environ- ment. We could create an environment where a real image is augmented with erroneous meanings (ie. “documents”), ie. incoherent with the real image, but this would be a minimum functionality. The potential of AR is best exploited in environments which for example propose diverting the primary function of integrating virtual objects in a real scene (second level of sub-functionality 3: perceptual association of the real and the virtual ). This is the case of the AR game developed by Dassault Systems for Nestlé [30] (Figure 1.8a), where a virtual character moves, partially concealing the real image, on a cereal box. Although this environment obeys objectives defined by a marketing strategy (eg. increasing traffic to Chocapic R ’s website and creating a strong emotional
En savoir plus

22 En savoir plus

Neuronal networks, spike trains statistics and Gibbs distributions

Neuronal networks, spike trains statistics and Gibbs distributions

One of the biggest ambitions in the field of spike train statistics is that techniques and ideas from statistical mechanics and thermodynamics will help us understand the collective “macroscopic” behavior of big populations spiking neurons with rela- tively few parameters. A recent paper ( Tkačik et al. , 2014 ), discuss about signatures of criticality found in retinal spike trains. The main idea is to use concepts and procedures from statistical mechanics considering networks with larger and larger numbers of neurons, expecting to see the emergence of a “thermodynamic limit" providing simpler universal behavior for the network as a whole, independent of microscopic details. The authors discover signatures of criticality by changing the parameters of the Maximum Entropy potential along one axis in parameter space (scaling). The fact that they observe a peak in the specific heat suggests that the real network is poised in the parameter space very close to a maximum in the vari- ance of log(probability), which constitutes the dynamic range of surprise that can be represented by the network. This peak is a signature of criticality (second order phase transition) in statistical physics. An important point discussed in this paper is that systems near critical points are maximally responsive to certain external signals, and this sensitivity may be functionally useful for the retina ( Tkačik et al. ,
En savoir plus

162 En savoir plus

Sampler trials on overconsolidated sensitive clay

Sampler trials on overconsolidated sensitive clay

This publication could be one of several versions: author’s original, accepted manuscript or the publisher’s version. / La version de cette publication peut être l’une des suivantes : la version prépublication de l’auteur, la version acceptée du manuscrit ou la version de l’éditeur. Access and use of this website and the material on it are subject to the Terms and Conditions set forth at

15 En savoir plus

Heights and representations of split tori

Heights and representations of split tori

 o Our main result on symmetries can be stated as follows: (cf. Theorem 3.2 for the precise statement) Theorem. Let E be one of the above G d m -modules. Then H E = W E . As the referee pointed out, our h E is a special case of a height function on toric varieties introduced by V. Malliot in [6] and later generalized by J. I. Burgos Gil, P. Philippon, and M. Sombra in [3]. This is evident if one compares our expression for h E given in formula (1.2), with the formula computing Malliot’s height [6, Corollary 8.3.2]. Our approach is rather more elementary than the sophisticated Arakelov geometry techniques employed by Maillot and later by Burgos Gil, Philippon, and M. Sombra. Also the questions addressed here are of a different nature than those taken up in [6] and [3] as we are mostly interested in computing the group H E .
En savoir plus

18 En savoir plus

Union and Split Operations on Dynamic Trapezoidal Maps

Union and Split Operations on Dynamic Trapezoidal Maps

Unite´ de recherche INRIA Lorraine, Technopoˆle de Nancy-Brabois, Campus scientifique, 615 rue du Jardin Botanique, BP 101, 54600 VILLERS LE` S NANCY Unite´ de recherche INRIA Rennes, Ir[r]

21 En savoir plus

GEOMETRIC ERGODICITY OF THE BOUNCY PARTICLE SAMPLER

GEOMETRIC ERGODICITY OF THE BOUNCY PARTICLE SAMPLER

ALAIN DURMUS, ARNAUD GUILLIN, PIERRE MONMARCH ´ E Abstract. The Bouncy Particle Sampler (BPS) is a Monte Carlo Markov Chain algorithm to sample from a target density known up to a multiplicative constant. This method is based on a kinetic piecewise deterministic Markov process for which the target measure is invariant. This paper deals with theoretical properties of BPS. First, we establish geometric ergodicity of the associated semi-group under weaker conditions than in [ 10 ] both on the target distribution and the velocity probability distribution. This result is based on a new coupling of the process which gives a quantitative minorization condition and yields more insights on the convergence. In addition, we study on a toy model the dependency of the convergence rates on the dimension of the state space. Finally, we apply our results to the analysis of simulated annealing algorithms based on BPS.
En savoir plus

35 En savoir plus

Statistics of spikes trains, synaptic plasticity and Gibbs distributions.

Statistics of spikes trains, synaptic plasticity and Gibbs distributions.

Following these tracks, we have investigated the dynamical effects of a subclass of synaptic plasticity rules (including some implementations of STDP) in neural networks models where one has a full characterization of the generic dynamics [33, 34]. Thus, our aim is not to provide general state- ments about synaptic plasticity in biological neural networks. We simply want to have a good mathemat- ical control of what is going on in specific models, with the hope that this analysis should some light on what happens (or does not happen) in “real world” neural systems. Using the framework of ergodic the- ory and thermodynamic formalism, these plasticity rules can be formulated as a variational principle for a quantity, called the topological pressure, closely re- lated to thermodynamic potentials, like free energy or Gibbs potential in statistical physics [35]. As a main consequence of this formalism the statistics of spikes are more likely described by a Gibbs proba- bility distributions than by the classical Poisson dis- tribution.
En savoir plus

7 En savoir plus

Probabilistic graph-coloring in bipartite and split graphs

Probabilistic graph-coloring in bipartite and split graphs

Dealing with approximation issues, we show that the 2-coloring (U, D) of a (connected) bi- partite graph B(U, D, E) achieves standard-approximation ratio 2, under any system of vertex- probabilities. Furthermore, we propose a polynomial algorithm achieving standard-approxima- tion ratio 8/7 under any system of vertex-probabilities. We also provide a polynomial time standard-approximation schema, under any system of vertex probabilities, for split graphs. Fi- nally, we show that, even under identical vertex-probabilities, probabilistic coloring cannot be solved by a fully polynomial time standard-approximation schema. On the other hand, deal- ing with differential approximation, we show that the differential ratio of the 2-coloring in a bipartite graph can be unbounded below, i.e., it can tend to 0, when we consider any system of probabilities, but it is bounded below (tightly) by 1/2 when vertex-probabilities are identical. We also prove that 8/7-standard-approximation algorithm for probabilistic coloring achieves tight differentia-approximation ratio 4/5, while under identical vertex-probabilities this algo- rithm achieves tight differential-approximation ratio 9/10. We finally show that under identical vertex-probabilities probabilistic coloring admits a DPTAS in split graphs.
En savoir plus

34 En savoir plus

The interacting 2D Bose gas and nonlinear Gibbs measures

The interacting 2D Bose gas and nonlinear Gibbs measures

3. Gibbs measures related to µ are known [5, 6, 17] to be invariant under suitably renormalized nonlinear Schr¨odinger flows. They also appear as long-time asymptotes for stochastic nonlinear heat equations, see [16, 18, 23] and references therein for recent results. 4. The above theorem is part of the more general enterprise of gaining mathematical understanding on positive-temperature equilibria of the interacting Bose gas. The ground state and mean-field dy- namics of this system are now well-understood, but rigorous works showing the effect of temperature seem rather rare [4, 8, 19, 20, 21, 25].
En savoir plus

4 En savoir plus

Augmented Lagrangian and differentiable exact penalty methods

Augmented Lagrangian and differentiable exact penalty methods

Augmented Lagrangian methods became quite popular in the early seventies but then yielded ground to algorithms based on Newton's method for solving the system of necessary opti[r]

14 En savoir plus

Weighted coloring on planar, bipartite and split graphs: complexity and approximation

Weighted coloring on planar, bipartite and split graphs: complexity and approximation

We study complexity and approximation of MIN WEIGHTED NODE COLORING in planar, bipartite and split graphs. We show that this problem is NP-complete in planar graphs, even if they are triangle-free and their maximum degree is bounded above by 4. Then, we prove that MIN WEIGHTED NODE COLORING is NP-complete in P 8 -free bipartite graphs, but polynomial for P 5 -free bipartite graphs. We next focus ourselves on approximability in general bipartite graphs and improve earlier approximation results by giving approximation ratios matching inapproximability bounds. We next deal with MIN WEIGHTED EDGE COLORING in bipartite graphs. We show that this problem remains strongly NP-complete, even in the case where the input-graph is both cubic and planar. Furthermore, we provide an inapproximability bound of 7/6 − ε, for any ε > 0 and we give an approximation algorithm with the same ratio. Finally, we show that MIN WEIGHTED NODE COLORING in split graphs can be solved by a polynomial time approximation scheme.
En savoir plus

25 En savoir plus

Show all 10000 documents...