Haut PDF Information in statistical physics

Information in statistical physics

Information in statistical physics

balian@spht.saclay.cea.fr Abstract We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involv- ing the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.
En savoir plus

33 En savoir plus

Physics and information : what is the role of information in physics?

Physics and information : what is the role of information in physics?

2.3 Quantum Information Theory After having had a brief excursion into the history of both quantum theory and information the- ory, let us look at how they started to interact with each other. This happened surprisingly late. Although they share common founders, such as for example John von Neumann, who substantially contributed to both fields, it took decades for scientists to connect them. Maybe because most of the brain power and physical resources were devoted to exploit the two new theories to the maximum. The development of new small transistors accelerated the construction ever faster and powerful computers while physicists convinced formerly hostile world powers to collaborate in the Conseil Europ´een pour la Recherche Nucl´eaire (CERN). This huge collaboration was meant to level the playing field under the premise that the possibility to construct another disastrous weapon like the nuclear bomb dormant in the depths of physics. One of the scientists working there in the late fifties was John Stewart Bell. As a hobby, as he put it [29], he was working on the foundations of quantum mechanics in his free time. He was especially interested in a ’paradox’ that was brought up by Einstein, Podolski and Rosen (EPR) in 1935 [30]. The basis of what EPR found to be para- doxical is a phenomena that known as entanglement (a term coined by Schr¨ odinger). Bell was able to follow EPR’s line of thought in order to derive his famous theorem and the inequalities that it implied. At that time his work did not have an immediate impact but we will see that it plays a central role for what will become quantum information theory.
En savoir plus

141 En savoir plus

Theoretical and numerical analysis of non-reversible dynamics in computational statistical physics

Theoretical and numerical analysis of non-reversible dynamics in computational statistical physics

2.1 Introduction Statistical physics gives a theoretical framework to bridge the gap between microscopic and macroscopic descriptions of matter [12]. This is done in practice with numerical methods known as molecular simulation [2, 64, 161, 101]. Despite its intrinsic limitations on spatial and timescales, molecular simulation has been used and developed over the past 50 years, and recently gained some recognition through the 2013 Chemistry Nobel Prize. One im- portant aim of molecular dynamics is to quantitatively evaluate macroscopic properties of interest, obtained as averages of functions of the full microstate of the system (positions and velocities of all atoms in the system) with respect to some probability measure, called thermodynamic ensemble. Some properties of interest are static (a.k.a. thermodynamic properties): heat capacities; equations of state relating pressure, density and temperature; etc. Other properties of interest include some dynamical information. This is the case for transport coefficients (such as thermal conductivity, shear viscosity, etc) or time-dependent dynamic properties such as Arrhenius constants which parametrize chemical kinetics.
En savoir plus

203 En savoir plus

Outstanding problems in the statistical physics of active matter

Outstanding problems in the statistical physics of active matter

dynamics is already out of equilibrium. This includes systems with absorbing states such as reaction diffusion processes [11] and the subject of this thesis, active matter. Absence of detailed balance prevents a description of these open systems in terms of a small number of phenomenological variable. This is why their study typically requires a more complicated framework, and the far from equilibrium world is still an active research topic in modern statistical mechanics. Because the equilibrium statistical physics toolkit is not relevant for those cases, several useful techniques have been developed over the years in order to solve them. We can cite for instance the Langevin and master equations, large deviations theory, or kinetic approaches such as the Bolztmann and Fokker-Planck equations from which hydrodynamic theories in terms of slow modes of the dynamics can be derived [12]. Moreover, remarkable progress in computing power in past decades now allow for both extensive numeri- cal studies directly at the microscopic level and numerical solving of the, typically nonlinear, equations that statistical physicists encounter everyday. It should finally be mentioned that a substantial part of nonequilibrium systems exhibit generic scale invariance [13]. Therefore, based on Renormalization Group ideas, the concept of uni- versality has also naturally emerged out of equilibrium and several universality classes can be defined in this context [11, 14–17].
En savoir plus

147 En savoir plus

Statistical physics and approximate message passing algorithms for sparse linear estimation problems in signal processing and coding theory

Statistical physics and approximate message passing algorithms for sparse linear estimation problems in signal processing and coding theory

which is thus a direct measure of the correlation between the matrices. Constructing a maximally incoherent sensing matrix for a given sparsifying basis Ψ is a computationally very hard problem and cannot be solved in general. But here the intuition suggesting that a purely random sensing matrix (that we will always take i.i.d Gaussian) must be highly incoherent with any Ψ with high probability (i.e. tends to one as the matrices size diverge) is valid. Indeed, drawing a random i.i.d Gaussian matrix will give rise to basis vectors uncorrelated with the Ψ ones, exactly as a white noise has a flat spectrum in any basies. This is one among many others advantages of working in high dimensions. It is quite instinctive to see that in a very high dimensional space parametrized by some basis, if you draw a random vector, it has very low probability to be close to aligned to one of the vector basis as there are so many available directions. Working with random Gaussian i.i.d matrices has another great advantage: it allows to perform analytical predictions in the large size limit N → ∞ using techniques mainly based on "law of large numbers like" arguments, standing at the roots of the state evolution analysis, see sec. 5.3.
En savoir plus

285 En savoir plus

Cell assemblies in neuronal recordings : identification and study through the inference of functional network models and statistical physics techniques

Cell assemblies in neuronal recordings : identification and study through the inference of functional network models and statistical physics techniques

point of view in [ 42 ]. This association can be accompanied by a reshaping of cell assemblies in the presence of effective inhibitory couplings, e.g. in session C. Last of all, if the same cell assembly is encountered in Task and in Sleep Pre, then it is conserved in Sleep Post, see session D in Fig. 3.37 . Despite the variability of the possible scenarios, Task cell assemblies (when present) are always reactivated in the Sleep Post epoch: this is in agreement with the finding that a session either does not show any significative mean potentiation or depression of the couplings from Sleep Pre to Sleep Post (Adj within one standard deviation from the average for the null model) or it shows a Task-related change of the couplings (Adj above one standard deviation from the average for the null model); potentiation or depression are never anticorrelated with the Task, as reflected by the absence of sessions with large and negative Adj. Some task-related cell assemblies, which are replayed during subsequent sleep, like the Replay groups of sessions A, B and G, are maximally or solely coactivated at the end of the experimental maze in successful trials, and seem therefore to be elicited by reward. This result does not seem to depend on whether the rat has learned the rule or not, since in session A the rule has just been changed and the rat does not perform better than chance; in session B (recorded from a different rat) the animal is progressively learning and, at the end of the session, it starts to perform better than chance; in session G (recorded from the same rat the day after session B) the rule is finally acquired and changed. The sporadic coactivations observed at the end of the maze in unsuccessful trials suggest that these Replay cell assemblies may respond not only to the presence of reward, but also (even if less strongly) to the expectation of reward. These empirical rules on cell assembly modification across the epochs and this intuition about a possible (certainly not the unique) meaning of the Replay cell assemblies in information coding in prefrontal cortex could be further validated and extended by the analysis of data sets with more (hundreds of) recorded neurons, which are beginning to be collected with the most recent multi-electrode techniques.
En savoir plus

189 En savoir plus

Application of information technology and statistical process control in pharmaceutical quality assurance & compliance

Application of information technology and statistical process control in pharmaceutical quality assurance & compliance

In fact the procedure for performing process capability analysis of Quality Assurance Data is titled Process Capability Analysis using Minitab@ Statistical Software. While t[r]

67 En savoir plus

Statistical Analysis to Establish the Importance of Information Retrieval Parameters

Statistical Analysis to Establish the Importance of Information Retrieval Parameters

This is why search engines are widely used tools when one needs to find accurate information. Indeed, they allow users to express their question, through a query submitted to a search engine, and retrieve a list of documents that might be of interest for this specific query. Therefore, IR can be defined as a ”set of tools and techniques that enable to find documents containing information that is relevant to a need” [Radhouani, 2010]. This represents the context of our work. IR is a very active research area that aims at proposing and evaluating the in- dexing, matching and ranking methods to retrieve relevant documents according to users’ queries. Many models have been defined in the literature, most of them having various parameters. To evaluate a model, the IR community relies on international benchmark collections, such as the ones from the Text REtrieval Conference (TREC) 1 , composed of sets of documents, topics, relevance judg-
En savoir plus

24 En savoir plus

Mutual information analysis: higher-order statistical moments, efficiency and efficacy

Mutual information analysis: higher-order statistical moments, efficiency and efficacy

quired when the leakage is embedded on higher-order statistical moments in presence of some countermea- sures (e.g. fourth-order in case of RSM with specific mask distribution) due to a need of an accurate PDF estimation. As a result, a trade-off between flexibility and efficiency can be adopted by adversaries accord- ing to a prior knowledge on the leakage characteristics. More generally, this work provides a characterization of MIA’s efficiency/flexibility according to the resolution (based on tuning parameters) involved in nonparamet- ric PDF estimation methods. Interestingly, one should note that low resolution MIA gives very similar results than AoV. We have shown that MIA conducted fol- lowing these guidelines compares favorably with DPA [8, 25], CPA [12], LRA [15] and AoV [43, 5, 23, 46] and are even superior in some cases where the latters fail. The purpose of the distinguishing rule presented in this paper is mainly to formalize the intuition behind our results related to the PDF estimation step and pro- vide study cases for which an a priori accuracy-based approach is not the straightforward way to achieve ef- ficiency in SCA context. Note that the application of this rule depends on the a priori knowledge on the statistical moment embedding the leakage and finetune hyperparameters for some MI-based distinguishers ac- cording to the latter is not essential in practice since more general practical guideline can be drawn based on our conclusions. Through this work, we feel that vari- ous hyperparameters in the SCA contexts will be able to be set using this proposed rule and we believe that some benefits can be achieved in adapting the principles of statistical methods to the task at end: SCA in the present case. The formal investigation of our results to further talk about optimality for MIA should be an in- teresting perspective as well as formally define in which context MIA is more efficient than CPA.
En savoir plus

19 En savoir plus

Statistical reasoning with set-valued information : Ontic vs. epistemic views

Statistical reasoning with set-valued information : Ontic vs. epistemic views

The ontic view is advocated by Colubi et al. [8] in the statistical analysis of linguistic data. The authors argue that they are interested in the statistics of perceptions (see [3] for a general presentation and defense of this paradigm). One of their experiments deals with the visual perception of the length of a line segment expressed on fuzzy scale using a linguistic label among very small, small, medium, large, very large. The considered experiment involves a simultaneous double rating: a free-response fuzzy rating along with a linguistic one. The alleged goal is to predict the category that a person considers correct for the segment. The linguistic evaluation is performed to validate the classification process introduced in the paper. The precise length of the segment exists but it is considered by these authors to be irrelevant for the classification goal. These authors concede that to predict the real length from the fuzzy perceptions requires a different approach. While the linguistic labels can arguably be considered as ontic entities, the fuzzy rating with free format can hardly be so, as it really is a numerical rendering of the imprecision pervading the perception of an actual length.
En savoir plus

18 En savoir plus

Topology of statistical systems : a cohomological approach to information theory

Topology of statistical systems : a cohomological approach to information theory

categories that behave like the category of sets with respect to certain operations; they also formalize generalized logical theories. We refer the reader to [15,55,61,65]. The category of presheaves on a given category can be seen as a topos for the trivial topology (topologie grossière). For us the main examples are the presheaf topoi on S or S op for a given information structure (S, E ); we shall see that in these topoi the relevant notion of localization is always linked to marginalizations (also called coarse- graining in the physics literature). The advantage of the topological viewpoint for the study of information structures is that new geometrical intuitions become available, as well as very general algebro-geometrical tools. We did not explore the connections to logic, but this undoubtedly constitutes a very interesting problem.
En savoir plus

227 En savoir plus

From few-body atomic physics to many-body statistical physics : the unitary Bose gas and the three-body hard-core model

From few-body atomic physics to many-body statistical physics : the unitary Bose gas and the three-body hard-core model

DOI: 10.1103/PhysRevLett.117.225301 First predicted in 1970 [1] , the Efimov effect describes the behavior of three strongly interacting bosons when any two of them cannot bind. At unitarity, when the scattering length diverges, the three-body bound states are scale invariant and they form a sequence up to vanishing binding energy and infinite spatial extension. Efimov trimers had been intensely discussed in nuclear physics, but it was in an ultracold gas of caesium atoms that they were finally discovered [2] . To observe Efimov trimers, experiments in atomic physics rely on Feshbach resonances [3] , which allow one to instantly switch a gas between weak inter- actions and the unitary limit. Such a control of interactions is lacking in nuclear physics or condensed matter experi- ments, and singular interactions can be probed there only in the presence of accidental fine tuning [4] . Beyond the original system [2] , Efimov trimers have now been observed for several multicomponent systems, including bosonic, fermionic, and Bose-Fermi mixtures [5–7] . These experimental findings are interpreted in terms of the theory of few-body strongly interacting quantum systems. For three identical bosons in three dimensions, a complete universal theory is available, on and off unitarity [4] . Further theoretical work is aimed at understanding bound states for more than three bosons, mixtures, and the effects of dimensionality.
En savoir plus

205 En savoir plus

Statistical reasoning with set-valued information : Ontic vs. epistemic views

Statistical reasoning with set-valued information : Ontic vs. epistemic views

2. Ontic vs. epistemic sets A set S defined in extension, is often denoted by listing its elements, say, in the finite case { s 1 , s 2 , . . . , s n } . As pointed out in a recent paper [33] this representation, when it must be used in applications, is ambiguous. In some cases, a set represents a real complex lumped entity. It is then a conjunction of its elements. It is a precisely described entity made of subparts. For instance, a region in a digital image is a conjunction of adjacent pixels; a time interval spanned by an activity is the collection of time points where this activity takes place. In other cases, sets are mental constructions that represent incomplete information about an object or a quantity. In this case, a set is used as a disjunction of possible items, or of values of this underlying quantity, one of which is the right one. For instance I may only have a rough idea of the birth date of the president of some country, and provide an interval as containing this birth date. Such an interval is the disjunction of mutually exclusive elements. It is clear that the interval itself is subjective (it is my knowledge), has no intrinsic existence, even if it refers to a real fact. Moreover this set is likely to change by acquiring more information. The use of sets representing imprecise values can be found for instance in interval analysis [54] . Another example is the set of models of a proposition in logic, or a propositional knowledge base: only one of them reflects the real situation; this is reflected by the DNF form of a proposition, i.e., a disjunction of its models, each of which is a maximal conjunction of literals.
En savoir plus

19 En savoir plus

On Probabilities in Biology and Physics

On Probabilities in Biology and Physics

chances constrain subjective probabilities according to the principal principle (PP); chances have the values that are assigned by rules which are part of a best system (BS) of laws and rules, striking as good a balance as the actual events will allow of simplicity, strength and fit; and (given the criteria of simplicity, strength and fit) chances supervene on everything that actually happens, the so-called ‘Humean mosaic’. But unlike in Lewis’ theory, in Frigg and Hoefer’s theory non-trivial chances may exist in deterministic universes. Following Hoefer ( 2007 , 558–559), Frigg and Hoefer argue that Lewis’ commitment to the existence of non-trivial chances only in indeterministic universes results from his view that all chances follow from the laws of fundamental physics. (This is reflected in Lewis’ reading of PP, where fundamental laws are always admissible information and accordingly chances in deterministic universes are trivial.) Frigg and Hoefer reject this view. They maintain that rules for assigning chances can be formulated in terms pertaining to different levels of discourse, such as microphysics, macrophysics, chemistry, genetics, mechanical engineering and meteorology, and that rules from all these disciplines have equal right to be considered for inclusion in a BS package. They then apply this revised version of THOC to CSM. There are two major traditions in CSM, sometimes seen as two different theories: one originates in the work of Gibbs and the other in the work of Boltzmann (Frigg 2008 ). Frigg and Hoefer argue that the probabilities in Boltzmannian CSM can be interpreted along their version of THOC. The problem of reconciling the expressly time-asymmetric behavior of irreversible thermodynamic processes with the underlying time-symmetric dynam- ics of CM systems can be related to the question of whether we can take our memory and records to be reliable. Suppose, for example, a glass of water with a half-melted ice cube in it, suitably isolated
En savoir plus

27 En savoir plus

Inference and modeling of biological networks : a statistical-physics approach to neural attractors and protein fitness landscapes

Inference and modeling of biological networks : a statistical-physics approach to neural attractors and protein fitness landscapes

Cognitive maps may be retrieved when the animal explores again the corresponding environments, or be quickly and intermittently recalled depending on the most relevant behavorial information at that moment [141]. Different sources of inputs to the hippocam- pus concur to form, recall, and dynamically maintain cognitive maps [202]. Changes in visual cues and landmarks may substantially affect place field shape and positioning [84]. The Path Integrator (PI), capable of integrating proprioceptive, vestibular and visual flow inputs and possibly supported by the grid-cell network in the medial-enthorinal cortex (mEC) [109], allows the animal to update the neural representation during naviga- tion [116]. The path integrator is itself sensitive to other sources of inputs, and undergoes reset in case of large disagreement with external landmarks or sensory information [126]. Insights about how these different inputs contribute to hippocampal representations were recently obtained by studying the effects of mismatches between path-integration and visual sensory information, in particular in virtual reality settings [124, 203]. In another study Jezek et al showed how abrupt changes in visual context (light conditions) during active exploration by a rodent resulted in fast flickering between context-associated maps in CA3 on the theta time scale [143] (Fig. 6.1A). Though they are largely artificial, these conditions offer a rare window on the fast dynamics of the place-cell population, and on how this dynamics is shaped by the various inputs.
En savoir plus

229 En savoir plus

Reactive Programming of Simulations in Physics

Reactive Programming of Simulations in Physics

then a second, and so on, until termination 3 . The only required property of instants is convergence: all the parallel components terminate at each instant. Execution of instants does not necessarily take the same amount of real time; actually, real time becomes irrelevant for the logic of simulations: the basic simulation time is the logical time, not the real time. The numerical resolution method works on a time discretized in time-steps during which forces integration is performed according to Newton’s second law. Typically, in simulations of atoms, time-steps have a duration of the order of the femto-second. Several steps of execution may be needed by the resolution algorithm to perform one time-step integration; for example, two steps are needed by the velocity-Verlet integration scheme [ 18 ] to integrate forces during one time-step: positions are computed during the first step, and velocities during the second. Actually, a quite natural scheme associates two instants with each time-step: during the first instant, each component provides its own information; the global information produced during the first instant is then processed by each component during the second instant. Note that such a two-instant scheme maps quite naturally to the velocity-Verlet method: each step of the resolution method is performed during one instant.
En savoir plus

22 En savoir plus

Computational and statistical challenges in high dimensional statistical models

Computational and statistical challenges in high dimensional statistical models

formulas for the MMSE. These formulas were first obtained heuristically using the replica method from statistical physics [Tan02, GV05] and later proven rigorously in [RP16, BDMK16]. Finally, another line of work has provided nearly-optimal computationally efficient methods in this setting using Approximate Message Passing [DMM09, DJM13]. However, to our best of knowledge, most of the techniques used in the proportional regime cannot be used to establish similar results when 𝑘 = 𝑜(𝑝) (with notable exceptions such as [RGV17]). Although there has been significant work focusing also on the sublinear sparsity regime, the exact identification of both the computational and the statistical limits in the sublinear regime, remained an outstanding open problem prior to the results of this thesis. We provide below a brief and targeted literature review, postponing a detailed literature review at the beginning of each Chapter, and provide a high-level summary of our contributions.
En savoir plus

301 En savoir plus

Fusion of statistical and structural information for flowchart recognition

Fusion of statistical and structural information for flowchart recognition

a human can say that these boxes were not the right ones to choose. We need to give to our method the ability to reproduce this choice, that is to be able to describe a good box. To construct boxes, we use line segments extracted through polygonal approximation. The finally built boxes use the whole strokes, with no segmentation. Furthermore, we tolerate in our analysis the strokes that are longer or shorter than the expected edges to deal with handwritings. Figure 3 shows two examples where strokes inside and outside a quadrilateral were used to construct a box. A human operator would not have chosen these quadrilaterals as boxes because they differ too much from initial strokes. We desire here to have a measure able to quantify this deformation:
En savoir plus

6 En savoir plus

Robust statistical registration of 3D ultrasound images using texture information

Robust statistical registration of 3D ultrasound images using texture information

As it is mostly non-invasive and has a real time capabil- ity and a relatively low cost nature, 2D ultrasound is pop- ular. Its major drawback is its weak capability of issuing quantitative accurate morphometric information [1]. In fact, conventional ultrasound exams are limited by 2D viewing, and follow-up studies are then not easily reproducible. 3D ultrasound imaging overcomes these limitations. In addi- tion, it facilitates extensive investigation and allows accu- rate measurements of organ volumes.

5 En savoir plus

Statistical Information Propagation Across Operators for Dynamic Power Estimation on FPGAs

Statistical Information Propagation Across Operators for Dynamic Power Estimation on FPGAs

the power consumption in the design flow of such systems and estimate power in various design stages. The power estimation techniques in FPGA can be divided into two categories according to the abstraction level of the circuit: low level (physical up to RTL) and high level or system level. At low level, transistors, logic gates and registers are specified and fully described physically, whereas at high level, only a global structure or behavioral view is considered. Generally, this last level lacks from techno- logical details which are crucial to get an accurate infor- mation on the dissipated power. Accurate power estima- tion is achieved at low-level with a significant simulation time that is often prohibitive.
En savoir plus

5 En savoir plus

Show all 10000 documents...