on the occasion of his 70th birthday
STOCHASTIC HOPFIELD NETWORKS WITH MULTIPLICATIVE NOISE AND JUMP MARKOV PARAMETERS
ADRIAN-MIHAIL STOICA and ISAAC YAESH
A non symmetric version of Hopfield networks subject to state-multiplicative noise, pure time delay and Markov jumps is considered. Such networks seem to arise in the context of visuo-motor control loops and may, therefore, be used to mimic their complex behaviour. In this paper, we adopt the Lur’e-Postnikov systems approach to analyze the stochastic stability and theL2 gain of generalized Hopfield networks including these effects.
AMS 2000 Subject Classification: 93E15, 93E20, 92B20.
Key words: stochasticH∞control, Hopfield neural network, recurrent neural net- work,S-procedure, linear matrix inequality, time delay.
1. INTRODUCTION
Hopfield networks are symmetric recurrent neural networks which exhibit motions in the state space which converge to minima of energy. Symmetric Hopfield networks can be used to solve practical complex problems such as implement associative memory, linear programming solvers and optimal guid- ance problems. Recurrent networks which are non symmetric versions of Hop- field networks seem to play an important role in understanding human motor tasks involving visual feedback (see [2]–[3] and the references therein). Such networks seem to be subject to effects of state-multiplicative noise, pure time delay (see [10], [11] and [16]) and even multiple attractors which can be caused by Markov jumps. In this paper, we adopt the Lur’e-Postnikov systems ap- proach to analyze the stochastic stability andL2 gain of generalized Hopfield networks including the effects of state-multiplicative noise and Markov jumps.
The paper is organized as follows. In Section 2, the problem is formulated while in Section 3 Linear Matrix Inequalities (LMIs) based conditions are de- rived forL2gain analysis. The case of a fixed and known pure time delay which is encountered in [2]–[3] is considered, using Pad´e approximations, in Section 4
MATH. REPORTS9(59),1 (2007), 111–122
to derive stability and disturbance attenuation properties of the stick balanc- ing dynamics. Finally, Section 5 includes concluding remarks. Throughout the paper Rn denotes the n dimensional Euclidean space, Rn×m is the set of all n×m real matrices, and the notation P > 0, (respectively, P ≥ 0) forP ∈ Rn×n means thatP is symmetric and positive definite (respectively, semi-definite). Throughout the paper (Ω,F, P) is a given probability space;
the argument θ∈Ω will be suppressed. Expectation is denoted byE{·} and conditional expectation ofx on the eventθ(t) =iis denoted byE[x|θ(t) =i].
2. PROBLEM FORMULATION
The neural network proposed by Hopfield, can be described by an ordi- nary differential equation of the form
(2.1) v˙i(t) =aivi(t) + n j=1
bijgj(vj(t)) + ¯ci =κi(v), 1≤i≤n,
where vi represents the voltage on the input of the ith neuron, ai < 0, 1 ≤ i ≤ n, bij = bji and the activations gi(·), i = 1, . . . , n are C1-bounded and strictly increasing functions. This network is usually analyzed by defining the network energy functional
(2.2) E(v) =− n
i=1
ai
vi
0
udgi(u) du du−1
2 n i,j=1
bijgi(vi)gj(vj)− n i=1
¯ cigi(vi), where it can be seen that dEdt = −dgi(vi)
dvi κi(v)2 ≤ 0, where the zero rate of the energy is obtained only at the equilibrium points, also referred to as attractors, where
(2.3) κi(v0) = 0, 1≤i≤n.
However, the neural network may be subject to environmental noise and to connection matrix perturbations which can be modelled as n
j=1dijwj(t) added to the right hand side of (2.1). The network subject to the combination of these two effects can be then described in matrix form as
(2.4) v(t) =˙ Av(t) +Bg(v) +Dw(t) + ¯C, 1≤i≤n, where
A:= diag(a1, . . . , an), B := [bij]i,j=1,...,n, C¯ :=
¯
c1 ¯c2 · · · ¯cn T , v:=
v1 v2 · · · vn T and g(v) :=
g1(v1) g2(v2) · · · gn(vn) T
The stochastic version of this network driven by white noise, has been considered in [10] where the stochastic
stability of (2.1) has been analyzed and where it has been shown that the network is almost surely stable when the condition dEdt ≤ 0 is replaced by LE ≤ 0, where L is the infinitesimal generator associated with the Itˆo type stochastic differential equation (2.4). This condition has been shown in [10] to be only satisfied in cases where the driving noise in (2.1) is not persistent. This non persistent white noise can be interpreted as a white-noise type uncertainty in A and B, but it does not infer any stability results for the practical case of real uncertainties. Hopfield [15] considered networks with Markov jump parameters. Encouraged by the insight gained in [2] and [3] regarding the role of state-multiplicative noise and time delay (see also [13]) in visuo-motor control loops, we generalize the results of [15] to include this effect. The Lur’e-Postnikov systems approach ([14], [1]) is invoked to analyze stability and disturbance attenuation (in theH∞norm sense) and the results are given in terms of Linear Matrix Inequalities (LMI).
To analyze the effect of w(t) we first define the error of the Hopfield network output with respect to its equilibrium points by
(2.5) x(t) =v(t)−v0.
and assume that the errors vectorx(t) satisfy
dx= (A0(θ(t))x+B0(θ(t))f(x) +Dw)dt+
(2.6)
+A1(θ(t))xdη+B1(θ(t))f(x)dξ, where the system output is
(2.7) z=L(θ(t))x
andη(t) andξ(t) are mutually independent standard Wiener processes. Note that (2.6) was obtained from (2.4) by replacingAdtby A0dt+A1dξ,Bdtby B0dt+B1gdξ and f(x) = g(x+v0)−g(v0). Note that A0(θ(t)), A1(θ(t)), B0(θ(t)), B1, D(θ(t)) andL(θ(t)) are piecewise constant matrices of appropri- ate dimensions whose entries are dependent upon the modeθ(t)∈ {1, . . . , r}, wherer is a positive integer denoting the number of possible models between which the Hopfield network parameters can jump. Namely, A0(θ(t)) attains the values of A0,1, A0,2, . . . , A0,r, etc. It is assumed that θ(t), t ≥0 is a right continuous–time homogeneous Markov chain onD={1, . . . , r}with a proba- bility transition matrix
(2.8) P(t) = eQt; Q= [qij]; qii<0;
r j=1
qij = 0; i= 1,2, . . . , r.
Given the initial condition θ(0) = i, at each time instant t, the mode may maintain its current state or jump to another mode i = j. The transitions between ther possible states i∈ D may be the result of random fluctuations of the actual network components (i.e., resistors, capacitors) characteristics
or can used to artificially model deliberate jumps which are the result of pa- rameter changes in an optimization problem the network is used to solve. In visuo-motor tasks one may conjecture that proportional and derivative feed- backs are applied on the basis of time sharing, where transition probabilities define the statistics of switching between the two modes. Although there is no evidence for this conjecture, the model analyzed in the present paper can be used to check its stability andL2 gain. The model (2.6), (2.7) will be ana- lyzed in Section 3. To include the delay effect needed to derive stability and disturbance attenuation properties for the stick balancing problem of [2]–[3]
we will employ Pad´e (see e.g. [19]) approximations of the time-delay. Namely, the system
dx(t) = [A0(θ(t))x(t) +B0(θ(t))f(x(t−τ)) +D(θ(t))w(t)] dt (2.9)
+A1(θ(t))x(t)dη(t) +B1(θ(t))f(x(t−τ)) dξ(t−τ), z(t) =L(θ(t))x(t),
(2.10)
whereξ(t) is another standard Wiener process independent of η and τ is the pure time delay, will be replaced by
dx(t) = [A0(θ(t))x(t) +B0(θ(t))f(δ(t)) +D(θ(t))w(t)] dt (2.11)
+A1(θ(t))x(t)dη(t) +B1(θ(t))f(δ(t)) dξ(t), z(t) =L(θ(t))x(t),
(2.12)
where δ(t) is the Nth order Pad´e approximation to x(t−τ). Choosing the so called diagonal Pad´e approximation (see [19]) the i element xi(t−τ) is approximated by the outputδi(t) of the system feeded byxi(t) and where the transfer-function of which is
hi(s) = N
k=0Ansn N
k=0Bnsn.
The coefficients in this equation whereB0 = 1 can be calculated using [19]:
Φ
B1
B2
... BN
=−
a1 a2 ... aN
, where Φij = aN+i−j and n
k=0aksk is the nth order power series of e−sτ. The coefficients A0, . . .AN are then obtained by Ak =k
j=0ak−jBj. Taking N = 1, the first order Pad´e approximation to e−sτ is just hi(s) = 1−1+sτ /2sτ /2. Namely, δi = 2ζi −xi, where ˙ζi = −2τζi + τ2xi. The Pad´e approximation allows to express the time-delay equation (2.11) in the form (2.6) without delay but rewritten in the extended state space [xT ζT]T where ζ = [ζ1, . . . , ζn]T.
However, this representation of the delay requires dependence off(·) on linear combinations of the state-vector components, rather than the components.
We, therefore, reformulate (2.6), (2.7) as
dx= (A0(θ(t))x+B0(θ(t))f(y) +Dw)dt+
(2.13)
+A1(θ(t))xdη+B1(θ(t))f(y)dξ, where the system output is
(2.14) z=L(θ(t))x
and
(2.15) y=C(θ(t))x.
Remind that in the time–delay case the state matrices are expanded with the corresponding dynamics of the Pad´e approximation. Only for the sake of sim- plicity we kept the above equations an identic notation with the case without delay. In the forthcoming analysis, we will assume that the components fi, i= 1, . . . , n off(y) satisfy the sector conditions
(2.16) 0≤yifi(yi)≤σiyi2 which are equivalent to
(2.17) −Fi(yi, fi) :=fi(yi)(fi(yi)−σiyi)≤0.
We shall further assume that
(2.18) ∂fi
∂yi ≤σi, i= 1, . . . , n.
Although the latter assumption of (2.18) further restricts the sector-type one class of (2.17), the applicability of our results remains since it is fulfilled by the usual nonlinearities as saturation, sigmoid, etc., used in the neural networks.
Some additional notation is now in place. Define S= diag
σ1, σ2, · · · , σn , whereσi are the nonlinearity gains of (2.17).
As mentioned above, we shall analyze stability (in stochastic sense) con- ditions for systems (2.6), (2.7) and (2.11), (2.10), respectively, together with theL2 gain boundness condition expressed as
(2.19) J =E ∞
0
zT(t)z(t)−γ2wT(t)w(t) dt
<0, x(0) = 0.
3. L2 GAIN ANALYSIS
Introduce the Lyapunov-type function (3.20) V(x(t), θ(t)) =xT(t)P(θ(t))x(t) + 2
n k=1
λk
Ckx
0
fk(s) ds depending on the nonlinearitiesfi(yi) =fi(Cix) via the constantsλi, whereCi
is theith row inC. As was mentioned in [1],V defines a parameter-dependent Lyapunov function. To see this, consider the simple case offi(xi) =xiσi and getV(x, σ1, σ2, . . . , σn) =xT(P+S12ΛS12)xwhich depends on the parameters σi, i = 1,2, . . . , n and on the constants λi, i = 1,2, . . . , n via (3.23) in the sequel. Applying the Itˆo-type formula (see [5], [8]) for V (x, θ), it follows that
E{V(x, θ(t)|θ(0))} −E{V (0, θ(0)|θ(0))}=E t
0
LV (x, θ(s)) ds
, where
LV(x, θ) := (A0(θ)x+B0(θ)f(x) +D(θ)w)T∂V
∂x +xTAT1 (θ)
P(θ) + diag
λ1∂f1
∂x1, . . . , λn∂fn
∂xn
A1(θ)x +fTB1T(θ)
P(θ) + diag
λ1∂f1
∂x1, . . . , λn∂fn
∂xn
B1(θ)f+ r j=1
qijxTPjx
and where for simplicity we have used the notationf :=f(y(t)). Then condi- tion (2.19) is satisfied if
(3.21) LV ≤γ2wTw−zTz, which becomes
xTAT0i+fTB0iT+wTDiT Pix+CTΛf + (3.22)
+
xTPi+fTΛC
(A0ix+B0if+Diw) + +xTAT1i
Pi+ diag
λ1∂f1
∂x1, . . . , λn∂fn
∂xn
A1ix+
+fTB1iT
Pi+ diag
λ1∂f1
∂x1, . . . , λn∂fn
∂xn
B1if+ +
r j=1
qijxTPjx+xTLTi Lix−γ2wTw≤0, i= 1, . . . , r, where
(3.23) Λ := diag
λ1, λ2, · · ·, λn
.
In order to explicitly express (3.22), assumption (2.18) will be used. Indeed, using inequalities (2.18) it follows that conditions (3.22) are satisfied if the following inequalities are satisfied:
−Fi0(x, f) :=xT
AT0iPi+PiA0i+AT1i
Pi+CTS12ΛS12C
A1i+ (3.24)
+LTi Li+ r j=1
qijPj+γ−2PiDiDiTPi
x+
+fT
B0iTCTΛ+ΛCB0i+γ−2ΛCDiDTi CTΛ+B1iT
Pi+CTS12ΛS12C
B1i
f +fT
B0iTPi+ΛCA0i+γ−2ΛDiDTi Pi x+xT
PiB0i+AT0iCTΛ+γ−2PiDiDiTΛ f
−
γwT−γ−1xTPiDi−γ−1fTΛCDi γw−γ−1DiTPix−γ−1DTi CTΛf
≤0.
Using theS-procedure ([1]) one, therefore, obtains that (3.21) subject to (2.17) is satisfied if there existτi≥0,i= 1,2, . . . , n, such that
(3.25) Fi0(x, f)−
n k=1
τkFk(x, f)≥0.
Denoting
(3.26) T := diag
τ1, τ2, · · · , τn and noticing that
− n k=1
τkFk(x, f) = n k=1
τkfk2−τkσkfkyk=fTT f−1
2fTT CSx−1
2xTSCTT f, (3.25) yields
xT
AT0iPi+PiA0i+AT1i
Pi+CTS12ΛS12C
A1i+LTi Li+ r j=1
qijPj+ +γ−2PiDiDiTPi
x+fT
B0iTPi+ ΛCA0i+γ−2ΛCDiDTi Pi+1 2T CS
x+
+xT
PiB0i+AT0iCTΛ +γ−2PiDiDiTCTΛ + 1 2SCTT
f+ +fT
B0TiCTΛ + ΛCB0i+γ−2ΛCDiDiTCTΛ+
+B1iT
Pi+CTS12ΛS12C
B1i−T
f ≤0,
i= 1, . . . , r. These conditions are satisfied if (3.27)
Zi11 Zi12 PiDi
Zi12T Zi22 ΛCDi
DiTP DiTCTΛ −γ2I
≤0, i= 1,2, . . . , r, where
Zi11:=AT0iPi+PiA0i+AT1i
Pi+CTS12ΛS12C
A1i+LTi Li+ r j=1
qijPj, (3.28)
Zi12:=PiB0i+AT0iCTΛ +1
2SCTT, Zi22:=BT0iCTΛ+ΛCB0i+B1iT
Pi+CTS12ΛS12C
B1i−T, i= 1, . . . , r, with the unknown variables Pi,Λ and T. The above developments are con- cluded in the following result:
Theorem 1. System (2.6), (2.7) is stochastically stable and its L2 gain is less thanγ >0if there exist symmetric matrices Pi >0, i= 1, . . . , r,Λ>0 andT >0 of the form(3.23)and (3.26), respectively, satisfying the system of LMIs(3.27) with notation (3.28).
4. EXAMPLE – STICK BALANCING
In [4] the problem of human stick balancing was considered. Actual test data showed that the task of stick balancing which is essentialy an inverted pendulum control problem is performed by humans using the combination of delay and state-multiplicative noise. The model
(4.29) φ˙˙=−Γ ˙φ+qsin(φ) +f(−k(θ)(R0+R1ν)φ(t−1))
was used in [4] to explain the test results, where Γ = 3¯mγ,q= 3gτr2,c= 3τmr2 and m and are stick mass and length respectively, τr is the delay and ν is the state-multiplicative noise. The time in model (4.29) is normalized with respect to the delay τr. For the stability analysis in the sequel we take sin(φ) ≈ φ.
We also take f(x) = 1000 tanh(x/1000). Defining the state-vector of (4.29) by x =
φ φ˙ T
we obtain the state-space description of (2.10), where A0 =
0 1 q −Γ
, B0 =
0 0 k(θ)R0 0
and B1 =
0 0 k(θ)R1 0
, assuming that|φ| 1. We also takeA1 =
0 0 0 0
andD= 0
1
. The nonlinearities are f1(x1) = 1000 tanh(x1/1000) and f2(x2) = 0. We also assume for this model thatk(θ) = 1, namely this model does not involve Markov jumps. We also note that we took a nonzeroDjust for the sake of illustrating computation
of the disturbance attenuation factorγ but for simulations we assumedw= 0.
It is pointed out in [4] that corrective movements of the inverted pendulum tip (i.e., the tip of the stick) occur also at time scales shorter than the delay. This phenomenon is referred to in [4] as on-off intermittency and seems to have a stabilizing effect. Namely, the noise statistically drivesφto zero, and since the noise is state-multiplicative, its effect near origin reduces and the pendulum is somewhat noisily stabilized. In [4] the stability of the inverted pendulum model in the presence of delay and state-multiplicative noise was studied using simulations. Our results may provide an analytic tool to predict this stability including the state-multiplicative noise using the Pad´e approximation for de- lay. To illustrate our results, we takeτr = 0.07, R1 = 10, ¯γ = 100, m= 0.035 and = 0.62. Since the above model is written in terms of time normalized with respect to τr, we take τ = 1 for our analysis. To include the delay ef- fect, we choose the first order Pad´e approximation. We, therefore, redefine the state-vector of (4.29) to be x =
φ φ˙ ζ ζ˙ T
and obtain the aug- mented state-space description of (2.12), where A0 =
0 1 0 0
q −Γ 0 0
2 0 −2 0
0 2 0 −2
,
B0 =
0 0 0 0
0 0 k(θ)R0 0
0 0 0 0
0 0 0 0
and B1 =
0 0 0 0
0 0 k(θ)R1 0
0 0 0 0
0 0 0
, assuming that
|φ| 1. We also take A1 =
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
and D=
0 1 0 0
. The nonlinear- ities aref1(y1) = 0, f2(y2) = 0, f3(y3) = 1000 tanh(y3/1000) and f4(y4) = 0.
We takeR0= 0.2266981, R1 = 10 and assume that the control is on (θ(t) = 1) and off (θ(t) = 2) according to the Markov chain onD={1,2}with a probabil- ity transition matrix from (2.8) withQ=
−α α β −β
, where α= 0.25, β = 0.125. It follows that P(t) = eQt= (αt+βt)−1
βt+µ(t)αt αt−µ(t)αt βt−µ(t)βt αt+µ(t)βt
, where µ(t) = e−(α+β)t. We take k(θ) = 3 for θ(t) = 1 and k(θ) = 0 for θ(t) = 2. In the steady state, P(t) =
1 3 2
3 1 3 2
3
, namely, control is off most of the time. Substituting the above parameter values in Theorem 1, we find that γmin = 0.025. Therefore, adding Markov jumps to the sys- tem still results in a stable dynamics. In Fig. 1a-d (φ,φ,˙ ∆z/ = cos(φ),
and u=−(R0+R1ν)k(θ(t))f(φ(t−1)) simulation results are depicted. The Markov jumps are best seen in Fig. 2, where the times between 100 and 150 are only shown. Both the LMI based analysis and simulation results indicate the stability of the stick balancing scheme also in the presence of Markov jumps between on-and-off states of the control. Although some of the numerical val- ues need, probably, to be tuned to realistically model human stick balancing, the results may encourage further research about the possible effect of Markov jumps on balancing tasks.
0 100 200 300 400 500
−0.5
−0.4
−0.3
−0.2
−0.1 0
Time [sec] a)
θ
0 100 200 300 400 500
−1.5
−1
−0.5 0 0.5 1
Time [sec] b)
dθ/dt
0 100 200 300 400 500
0.9 0.92 0.94 0.96 0.98 1
Time [sec] d)
∆ z / L
0 100 200 300 400 500
−1500
−1000
−500 0 500 1000
Time [sec] c)
u
Fig. 1
5. CONCLUSIONS
A class of stochastic Hopfield networks subject to state-multiplicative noise, where the network weights jump according a Markov process have been considered. Stochastic stability and disturbance attenuation analysis in anH∞ setup have been derived in terms of Linear Matrix Inequalities. The results have been illustrated by a stick balancing related example, where the delay in the control has been replaced by its Pad´e approximation.
100 105 110 115 120 125 130 135 140 145 150
−250
−200
−150
−100
−50 0 50 100 150 200 250
Time [sec]
u
Fig. 2
REFERENCES
[1] S. Boyd, El. Ghaoui, L. Feron and V. Balakrishnan,Linear Matrix Inequalities in System and Control Theory. Philadelphia, PA:SIAM, 1994.
[2] J.L. Cabrera and J.G. Milton, Human stick balancing: tuning Levy flights to improve balance control. CHAOS14(2004), 691–698.
[3] J.L. Cabrera, R. Bormann, C. Eurich, T. Ohira and J. Milton, State-dependent noise and human balance control. Fluctuation and Noise Letters4(2001), L107–L118.
[4] J.L. Cabrera and J.G. Milton,On-Off Intermittency in a Human Balancing Task. Phys- ical Review Letters89(2002),15.
[5] V. Dragan and T. Morozan, Stability and robust stabilization to linear stochastic sys- tems described by differential equations with Markovian jumping and multiplicative white noise. Preprint 17/1999, The Institute of Mathematics of the Romanian Academy.
[6] V. Dragan and T. Morozan,Stability and robust stabilization to linear stochastic systems described by differential equations with Markovian jumping and multiplicative noise.
Stochastic Anal. Appl.22(2002),1, 22–92.
[7] V. Dragan and T. Morozan,The linear quadratic optimiziation problems for a class of linear stochastic systems with multiplicative white noise and Markovian jumping. IEEE Trans. Automat. Control49(2004),5, 665–675.
[8] X. Fen, K.A. Loparo, Y. Ji and H.J. Chizeck, Stochastic stability properties of jump linear systems. IEEE Trans. Automat. Control 37(1992), 38–53.
[9] P. Gahinet, A. Nemirovsky, A.J. Laub and M. Chilali,LMI control toolbox for use with MATLAB. The Mathworks Inc., 1995.
[10] S. Hu, X. Liao and X. Mao,Stochastic Hopfield neural networks. Statistical pulse com- pression. J. Phys. A36(2003), 1–15.
[11] X. Liao, G. Chen and E.N. Sanchez, LMI Based approach to asymptotically stability analysis of delayed neural networks. IEEE Trans. Circuits Systems I Fund. Theory Appl.
49(2002), 1033–1039.
[12] X. Liao and X. Mao,Stability of stochastic neural networks. Neural Parallel Sci. Comput.
4(1996), 205–224.
[13] F. Mazenc and S.-I. Niculescu,Lyapunov stability analysis for nonlinear delay system.
Systems Control Letters42(2001), 245–251.
[14] A.I. Lure and V.N. Postnikov,On the theory of stability of control systems. Appl. Math.
Mech.8(1944),3. (Russian)
[15] A. Stoica and I. Yaesh,Hopfield networks with jump Markov parameters. WSEAS Trans.
Systems4(2005),4, 301–307.
[16] A. Stoica and I. Yaesh,Delayed Hopfield networks with multiplicative noise and jump Markov parameters. In: Proc. MTNS, 2006, Kyoto, Japan.
[17] C.Y. Kao and B. Lincoln,‘Simple stability criteria for systems with time-varying delays.
Automatica40(2004), 1429–1434.
[18] M. Kocvara and M. Stingl,PENBMI User’s Guide.www.penopt.com, 2005.
[19] C.M. Bender and S.A. Orszag,Advanced Mathematical Methods for Scientists and En- gineers. McGraw Hill, New York, 1978.
Received 1 September 2006 “POLITEHNICA” University of Bucharest Faculty of Aerospace Engineering Str. Polizu, No. 1, 011061 Bucharest, Romania
amstoica@rdslink.ro and Control Department IMI Advanced Systems Div.
P.O.B. 1044/77 Ramat-Hasharon, 47100, Israel
iyaesh@imi-israel.com
