Information transfer in the lattice gas on random lattices

HAL Id: jpa-00246397

https://hal.archives-ouvertes.fr/jpa-00246397

Submitted on 1 Jan 1991

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Information transfer in the lattice gas on random lattices

A. Kitaev

To cite this version:

A. Kitaev. Information transfer in the lattice gas on random lattices. Journal de Physique I, EDP

Sciences, 1991, 1 (8), pp.1123-1130. �10.1051/jp1:1991194�. �jpa-00246397�

Classification

Physics

Abstracts

05.20

Information transfer in the lattice _gas

_on

random lattices

A. Yu. Kitaev

L. D. Landau Institute for Theoretical

Physics, Academy

of Sciences of the U-S-S-R-, 117940,

Kosygina

Str. 2, Moscow, U-S-S-R-

(Received 5

April

1991,

accepted

25 April 1991)

Abstract. A lattice _gas model _{on a} random

graph

with vertex

connectivity

_p is studied.

Information~theoretical arguments are used to

verify

the TAP

equations

and to

identify

the transition temperature. ^{The transition from} the low

density phase

to _a

glass phase

is continuous _at small _p and discontinuous at

large

_p.

1. Introduction.

One of the earliest

approaches

to the

spin glass problem

_was

proposed by Thouless,

Anderson and Palmer

[I].

The TAP

approach

has

played

an

important

role in

understanding

the nature of the

spin glass phase.

An unusual feature of this method is that the TAP free energy

(as

a

function _{of m;}

=

(s~) ⁾

^{is valid}

^only

^for some restricted domain in the order

parameter

space while the rest of

(m;)

_space is

unphysical.

For the TAP free _{energy to} be

valid,

finite-size corrections to the mean-field

theory

should _converge. For the SK

model,

this

requirement implies

that

pair spin-spin

correlations _are small

[1],

N~ ^'

£ ^((s; m;)(s~ m~))~

^l

,,j

In this paper the TAP

approach

will be

applied

to a more

complex

model with finite

connectivity.

It will be shown that the

pair

correlation test is insufficient for

verifying

the TAP method. Global correlations between all

spins together

_are

important

in _{some cases.} An

account of these correlations is based _on the concept of information transfer. A

key

role in

our consideration will be

played by

so-called mutual entropy ^between a

particular spin

and all

spins

at

infinity.

We conclude that if this

quantity

is

equal

to _zero then the TAP free energy is valid.

The model to be studied is _a finite

temperature

^{version of the well known} combinatorial

optimization problem

_« maximum

independent

set »

[2].

^It can also be

interpreted

_{as a} lattice

gas model. Let _us consider _a

graph

r

consisting

of N vertices and _some set of

edges

E.

Suppose

that any ^vertex can be empty _or

occupied (by

_an

atom)

which is described

by

N

spins

_»

f,,

,

f

~r

(f;

=

0, ).

The Harniltonian is

merely

the number of _atoms

multiplied by

_some chemical

potential,

3C=-P if;. (1)

l124 JOURNAL DE PHYSIQUE I lNf 8

Besides that, there _{is an} infinite interatomic

repulsion~

that is the

following

hard restrictions

are

imposed

f, f~

₌ ^{0 for} any

edge (I,

_{/ ~} E

(2j

Note that these restrictions lead to frustration if the

graph

r contains odd

loops. Finally,

_we

specify

the

graph

r to be _a random

graph

with coordination number

p~p

~2 j More

precisely,

_one chooses with

equal probabilities

_any

graph

with N

vertices,

each connected to p

neighbours.

Calculation _{of any}

quantity

for _a

particular graph

should be followed

by sample averaging.

The coordination number _p is assumed _to be _constant while N tends _to

infinity.

This model is rather similar to the Viana

Bray

^model

[3]

^with

antiferromagnetic

bonds

only.

However, the

spin-spin

interaction in _our model is stronger,

especially

for

large

_p. Another distinction is that the vertex coordination number _p does _not

depend

on the _{vertex or}

sample.

There _{are two}

independent

parameters ^{in the model:} p and

a =

~ Small values of

T

a

correspond

to a low lattice gas

density

and _a weak interaction

(normal phase).

At

large

a the

system

^{condenses into} a

glass phase.

We shall concentrate on the normal

phase

and its

stability.

When

dealing

with

self-averaging quantities (free

_energy~

entropy~ etc.)

one can

speak

about _{a «}

typical sample

_» and thus avoid

explicit sample averaging.

For _a

particular graph

r _{one can} define _a distance between _two vertices I and

j

_as the

length

of the shortest

path

from I to

j.

A very

important

property ^of a

typical

_{sparse ~p}

= const., N

- co

graph

is that it is

locally organized

_as the Bethe lattice

(Caley tree).

Almost all

loops

consist of _more than

log~N edges. Therefore~

_{one can} consider _a

large enough (I

« radius

«log~N)

tree- structured

vicinity

of _{some vertex} and _use the information about the entire

graph

to

impose

reasonable

boundary

conditions _on this

piece

of the system. ^This

approach

^has been

already applied

to the Viana

Bray

model

[4].

2. TAP entropy end free energy.

The purpose of this section is to express the entropy ^{and the} free _energy in _terms of the thermal averages

(f,)

= m, for _a

particular graph

r. It is convenient to use a more

general

Hamiltonian

KIT

=

-jja, f,.

^The local fields _{a, are} to be

adjusted

to

produce

the

prescribed

values of m,. The entropies of _a

single spin (,

^and a pair of spins

f,

and f~

(f, _f~

=

0)

_are

S,

=-m,Inm,-(I-m,)In(I-m,j

(3j

$~ =-m,lnm,-m~Inm~-jl-m,-m~)In(I-m,-m,).

For the Bethe lattice, the entropy ^{and the free} energy are given

by

the formulae

S

=jjS,- jj

_AS,~

(4j

, <,1>~E

(

=

La,

_m, s

isj

where AS,~ =

S,

+ S~ S,~. The TAP

equations islim,

= a can be obtained from the free energy minimum condition.

To prove

equation (4),

let _us start with the system of N disconnected

spins

and add the

bonds step

by

step. One should also

readjust

_a,

through

this _{process to conserve}

m;,

Any

_new

edge

added connects two disconnected

parts

^of a current

graph

since the resultant

graph

r has _no

loops.

The

entropy change

due to

adding

_one

edge (I, j

is

s" s'

=

As;y

⁺

£ (w>'(f;, fy) w'(f;, fj)) B(f,, f~)

<,,<j

where w' and w" _are the

probability

distributions

off;

and

f~

^{before and after the}

edge

_was

added, B(f;, f j)

is the system

entropy

for fixed values

off,

and

fj. However,

the second term

vanishes since

ml'=m), mj'=mj

and

B(f;,f~)

_can be

decomposed

into the _sum

Bj (f~)

₊

B~(fj), Therefore,

_any

edge

contributes

AS~~ ^to the total

entropy

to

yield equation (4).

One _can try to extend this

proof

to the _case of _a

typical

_sparse

graph

r. The

key point

in the above consideration is the

possibility

to

decompose B(f,, f~ ^).

^Such a

decomposition

is

always possible, provided

_any

spin

^correlates with at most one

off;

^and

fj.

This condition also _ensures

that it is

possible

to

readjust

_a; and _a

j ^so that to _conserve all the averages m~. For the _case of

a

typical

_sparse

graph,

this condition

implies

that there _{are no}

long-range

correlations in the system.

It is rather obvious that

long-range

correlations _are absent for small _a

(normal phase).

Moreover,

the averages

(f;)

= m; are

locally

determined in this _case

(in

^the

glass phase

m; may

depend

_on the

global

lattice

structure).

The local environments of all vertices _are the same, so all m; are

equal

to _a

unique quantity

_m~

= m

~(

_a

).

The TAP entropy ^and

equations

become

~=

_-m~lnm.+

_~p-I)(I -m~)In(I-m~) (6)

~P/2)(1-2m*)in(1-2m~)

a = Inm~+

~p-I)In(I-m~)-pln (1-2m~). (7)

We conclude this section

by

_a self-consistent calculation of the correlation function

G;j

=

((f, _m;)(f~ m~))

under the

assumption

that the TAP free energy

(5)

is

valid, am;

~iY

"

$

^~

~'J

j

J

R;y

=

~

= ~

~~ (8)

m~ ^m; m~

l _m

~

~J

^~

i~. '- ^~

~

~ l

-~

m ~

~'J

where _{~__}

~

(l

^if

^{(I, j}

^{~ E}

'J 0 otherwise

A

simple

calculation based _on the Bethe lattice

approximation yields

m~ ~>J

~'J '~*~~ '~*~

l _m~ ~~~

where d;~ ^{is the distance between the} vertices I and

j. Equation (9)

is _an

approximate

_one.

Actually, G;j

^{includes the} contributions from all the

paths

from I to

j (not only

from the shortest

one). However, equation (9)

becomes

precise

in N

- co

limit, provided

the _sum

l126 JOURNAL DE PHYSIQUE I lNf 8

N~~jjG,)

_converges at

large

distances

d,,.

This requirement ^leads to

inequality

,,

~p

)(ni«,<(I m~))~

~ l _or

m~~m~ where _m~= (101

,p-I

+I

It is

tempting

to

identify

the

corresponding

value _a~ from

equation (7j

with _a

phase

transition

point.

The trouble is that the entropy obtained

by

_a substitution of m~ ^into

equation (6)

is

negative

for

large

_{p ~p}

~ 29 ).

Therefore~ pair spin-spin

correlations reveal _no

singularity

at the true critical value _a

= a~ if

a tends _{to a~} from below.

3. Information

propagation

on a tree.

In this section _more

complex

correlations will be studied. Let _us choose any

spin (denoted by 0)

and consider _a

large (of

radius _r »

I)

tree-structured

piece

of the lattice

(Fig. I).

We _are to

measure correlations between

fo

and all

spins beyond

^the selected part ^{of the lattice.} It is

obvious that if

to

^does not correlate with the system ^of _spins s at the

boundary

of the selected

piece

then it does not correlate with _more distant

spins

as well. This fact _can be

expressed

_in

terms of so-called mutual entropy ^{which is}

commonly

used in information theory

[5].

)

F>g I. -A tree-structured p>ece of the fait>ce.

Let _.~j and

~~ be discrete variables and

>I(xj~ xi)

be their

probability

measure. The

probability

_measures for the random variables xi and ~j considered

separately

will be denoted

by >vj(.~j)

^and >t,~(.~~). ^{The mutual} entropy

(mutual information) Ij~

^is defined _as follows

~12"~t +~2~~12

where

Sj~

₌

jj

_ii~

_(.<j,

_.<~ ln _it-

(.;j, .;~)

II

,, ^,~

Sj

=

jj

_w-j

(xi

In _w>

ii-ii Sj

=

jj

>I.z(.;~ In w~~(,<~)

< »

Ij~

is the average amount of information about _xi which _can be obtained

by

_a measurement of x~

(or

vice

versa).

The _most

important properties

of this

quantity

_{are as} follows

1.

Ii~

_m o.

2.

Ij~

=

0 if and

only

if xj and x~ ^are

statistically independent.

3.

Suppose

_xj and J~ interact with each other

through

_some intermediate variable J~, that is

w(xj,

_x~,

x;)

=

u(xj, x;)

_v

(x;, x~).

Then

Ij~

_w

Ij,, Ij~

w

1;~.

Property

3

being applied

to _our model _means that the mutual

entropy Io~

^between

to

and _{s can} be used _{as an upper} bound for all

long-range

correlations. For _a TAP free _energy

verification,

it is sufficient to show that

Io~

- 0 _{as r}

- co.

We start from the

assumption

that the

long-range

correlations _are absent. This allows to

describe the interaction of the selected

piece

with the rest

part

^of the lattice

by

_an effective

boundary

field _a~~~d. Then it is

possible

to

compute Io~

and check the

original assumption.

A calculation of _any

quantity

for the Bethe lattice involves _some recursion from the leaves

to the root. We _say that _a vertex

j

^{is situated above} a vertex I if the

path

from 0 to

j

_passes

through

I. Each vertex

(except

the leaves and the

root)

_{has p} I

neighbours

above and I

neighbour

below. The root vertex 0 has _p

neighbours

above. For any ^vertex I _{one can} consider the subtree

T; consisting

of I and all the vertices above I

(Fig. I),

Let

Z; (f;)

be the

partition

function of

T;

^calculated for _a fixed value

off,.

We shall also _use the notations

Z;

=

Z~(0)

₊

Z;(I),

_y~

=

Z~(I)/&~ (in particular,

_yo

=

(to)

=

mo).

The recurrent relations for all this

quantities

_{are very}

simple,

Z;(0)

=

fl (Zj(0)

+

Zj(I )) Z;(I)

= e~

fl Z~(0) ⁽¹²⁾

j ^-I

yi=

(I

^+e~~

^fl

j

~Yj

Here

j

runs over all the _nearest

neighbours

above the _vertex I. The

fix-point

for

y is

m~

y~ ₌

(13)

The

boundary

field _a~n~d is

self-adjusted

to

yield precisely

this value of y. Note that the fix-

point

is unstable if _m~

~

lip-

A small deviation of a~~~~d leads to

antiferromagnetic ordering

in the lattice domain under consideration.

However,

this

instability

is

suppressed

in the entire lattice due _to the _presence of odd

loops. Therefore,

_{one may}

merely ignore

^this

instability.

The next step ^{is to} freeze the

boundary spins

at random and to

study

the distribution of Y;,

f, ~y;)

=

z y,

~j~(j ^{'~ w~(s,) (14)}

~, ^s i

Here _s~ stands for the collection of the

boundary spins

which fall into

T;, Z,~(f;, s;)

is the

partition

function of

T;

for fixed

f;

and s;,

Z~(s,)

=

Z;~(0,s~)

+

Z;~(l,s;), w~(s;)

is the

probability

of _a

given boundary spin configuration.

We _use the thermal

boundary spin

distribution

w~(s;)

=

Z~(s;)/Z;, w;~(f;, s;)

=

Z;~(f;, s;) IT,

which makes _a diffierence from the

approach [4].

^The

probabilities wo~(fo, s)

_are

just

the

quantities

to be used for _a mutual

entropy calculation,

_{so one} obtains

1128 JOURNAL DE PHYSIQUE t4" 8

"

Hi. Hi

In,

" m in ₊ I _i?i in

f~(m

dm

(15)

m I _m

The functions

f, ii-,

^{have the}

following properties (for

I #

0)

~f(J') ^dJ'

= '

~

J'flJ')

dv

= ~

ii

6j

For I

=

0, _one should

replace

_y~ with _m~.

Recurrent relations for

Z,~(f,, s,)

and

y~(s,)

=

Z,~(l,

_s,

)/Z~ is,

^differ from

equations (12) only by

the presence of the additional function arguments

s~ and s, =

is,

~,s~~,

).

From these relations and

equations (14), (16)

_{one can} obtain the relation for

f,(v, ),

1>~

~

j ^'

f, _{(J'j )}

= j '~ ^{3 JJ, i + e}

fl fl f~ u

_j

j

d_>,~ i 7

j

j

J'i

j

where

j

_{runs over p-} I

neighbours

of the vertex I.

(For 1=0,

one should

replace

y~ and _p with _m~ and

p).

Assuming

all

f,

in

equation (17j

to be the _same

function,

_{one can} write this equation in the

p~3 _'~rn~

,

~0 11 42

ioc

l

nn~=O

414

~

2

nn~=04145

3

nn~=Q

415

4

nn,=Q

d16

' so

I

~

50

~

~

~

~d 40

~

i

20

, ,

/

~'~

~

^{~ '}

oo

00 02 04 05 08 10

rTl

Fig. 2. Local magnetization distribution for _p 3.

p=37 [m~ =0.1429)

o.o

m~=Q.127

2

m~=0.1275

3

m~=0.128

6.0

~

/~

~/

_p

4:0

2.0

3

O-O

0.0 0.4 0.6 O-B 0

m

Fig.

3. Local magnetization distribution for p =

37.

symbolic

form

f,=Afj,

^{where A is} a nonlinear

integral operator.

The

operator

A should be

applied repeatedly

to the initial function

finit(y)

"

(i y*)

3

(y)

_{+ y* 3}

(y 1) (18)

If this process converges to the trivial

fix-point f~y)

₌ 3

(y y~)

then the mutual

entropy Io~

^tends to _zero.

Numerical

computations

have been done for different p and _a. The restrictions

(16)

_were

imposed

to avoid _an undesirable

instability. Any

deviation from these restrictions _was

being

corrected

through

the

computation

_process

(after

each

iteration).

After

sufficiently

_many

iterations the process

always converged

to some

fix-point f~~y).

Then _a modified

operator Ao

was

applied

to this

fix-point

to obtain

fo(m ).

The results for p = 3 and p =

37 _are shown in

figures 2,

3

(different

curves are indexed with

m*(a )

rather than

a).

The transition from the trivial

fix-point

to _a nontrivial _one have different nature for small and

large

_p. For small

p, the critical value _aA is

equal

to a~. The

peak gradually

become wider _{as a}

(or

m~)

increases _so the transition _seems to be continuous in this _case. On the

contrary,

^for

large

p a

sharp

transition _occurs at aA < a~. The critical value p~

separating

_« small _» and _«

large

_»

p is found to be

greater

than 25 and less than 30.

(A

_more

precise

identification of p~

requires

_a modification of the numerical

algorithm

to reduce

rounding errors).

l130 JOURNAL DE PHYSIQUE I M 8

4. Discussion.

It is shown that information-theoretical arguments can be used to

verify

the TAP

approach

and _to

identify

the

temperature

^{of the}

normal-to-glass phase

transition. For small _p the transition is similar to the continuous transition in the SK model. For

large

_p~ the transition is

sharp

and _seems to be similar to that of the Potts

glasses [6, 7].

It _can be

interpreted

_as

ergodicity breaking.

Just above _a~ the

configurational

_space is subdivided into _many

weakly

connected parts ^which are refer _{to as} system states. For this _reason any ^{vertex I} is

characterized

by

_a distribution of m, =

(f,)

rather than _a

unique

value of m,. One _can

identify (at

least if _a

-

a()

this distribution with the function

fo(m)

calculated above.

However~ the Potts

spin

system

undergoes

two

phase

transitions _{as a} temperatures decreases

[7].

The first transition at T=

T~

is _a

purely dynamical

_one. An

analytical expression

for the free energy ^at

high

temperature ^{remains valid} up ^to the second transition

temperature

T~

~

T~.

As to _our

model,

_a finite mutual information between any

spin

and

infinity

does not

necessarily-

mean that the TAP

approach

is _not

applicable.

The

only physical

consequence one can

definitely

conclude in this _case is that there _are

long-range, long-time dynamical

correlations. The transition at a~ is

likely

_a

dynamical

_one while the free energy has _a

singularity

at a~ ~ a ~. In the intermediate

phase (a~

_{~ a ~ a}

~)

the TAP entropy

(6)

is

probably

valid but it _must be

distinguished

from the average entropy ^of a system state. The

difference S~~~j =

Sr~p £

_{is called the}

configurational

entropy

[7].

The number of _states which contribute _to the total free _energy is

exp(S~~~i).

The

quantity ^£

can be obtained

by averaging

the TAP entropy

(4)

_{over m,.} The result looks _as follows

~

=

(l p/2) lo

₊

_~p/2j (1 m,) S,

~

where

lo

= (- m In _m I _m In _m

fo(m

j dm

j19)

S~ = j- v In _i'- i'j In

jl i')j j,(>')

^di~.

The

configurational

entropy _per

spin

is

positive

at a~. It becomes _{zero at some} a~ ~ a~. So the number of states is

exponential (e~~)

in the intermediate

phase

and finite _or

polynomial

in the

ordinary glass phase (a

_~

a~).

Acknowledgments.

I wish to thank M.

Feigelman

and L. Levitov for

helpful

discussions on this work.

References

ill

THOULESS D. J., ANDERSON P. W. and PALMER R. G., Philos. Mag. 35 (1977) 593

[2] ^{GAREY M. R. and JOHNSON D.} S., Computers and

Intractability

(W. H. Freeman and

Company,

San Francisco, 1979).

[3] ^ViANA L. and BRAY A. J., J. Phys. C18 (1985) 3037.

[4] ^{CARLSON J.} M., CHAYES J. T., CHAYES L., SETHNA J. P. and THOULESS P. J., Europhjs. ^Lelt. 5

(1988)

355.

[5] For example, _see KoLmoGoRov A. N., Information Theory and

Theory

of Algorithms (Nauka, 1987) (in Russian).

[6] ^{GRoss D.} J., KANTER I. and SOMPOLINSKY H.,

Phys.

Rev Lett. 55 (1985) 304 [7] KIRKPATRICK T. R. and WOLYNES P. G., Ph_i~s. Re>. 836 (1987) 8552.